Jul 16 22:17:21.876503 ip-10-0-136-7 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.174401    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178963    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178977    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178980    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178983    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178987    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178990    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178994    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:17:24.300504 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178997    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:17:24.261080 ip-10-0-136-7 systemd[1]: Started Kubernetes Kubelet.
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.178999    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179002    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179004    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179007    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179010    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179012    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179015    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179018    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179020    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179023    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179026    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179028    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179031    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179033    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179036    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179039    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179041    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179044    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:17:24.302340 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179047    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179053    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179055    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179058    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179061    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179065    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179069    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179072    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179075    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179077    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179080    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179083    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179086    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179089    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179092    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179094    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179097    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179101    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179104    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:17:24.303063 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179108    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179113    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179119    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179122    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179125    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179128    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179149    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179153    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179156    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179159    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179161    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179164    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179168    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179171    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179180    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179183    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179186    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179188    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179191    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:17:24.310252 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179193    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179196    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179199    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179201    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179204    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179209    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179217    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179220    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179223    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179226    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179229    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179231    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179234    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179237    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179244    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179247    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179249    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179252    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179255    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179258    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:17:24.311105 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179260    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179263    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179265    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179268    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179270    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179274    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179276    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179279    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179288    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179291    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179294    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179297    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179299    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179302    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.179304    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183951    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183961    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183965    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183970    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:17:24.312809 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183974    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183978    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183982    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183987    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183990    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183993    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183996    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.183999    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184001    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184004    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184007    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184010    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184012    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184015    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184017    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184020    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184023    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184025    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184028    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184030    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:17:24.313687 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184033    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184035    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184039    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184041    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184043    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184046    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184049    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184052    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184054    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184057    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184060    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184062    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184065    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184068    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184070    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184073    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184075    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184078    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184080    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184083    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:17:24.314984 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184085    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184088    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184090    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184092    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184095    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184097    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184100    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184103    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184105    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184109    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184112    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184115    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184118    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184121    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184125    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184154    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184160    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184165    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184171    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184174    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:17:24.425243 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184177    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184180    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184184    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184187    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184189    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184192    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184194    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184197    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184200    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184202    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184205    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184207    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184210    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184213    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184216    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184219    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184221    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184224    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184227    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184229    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:17:24.426421 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184232    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184234    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184237    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184239    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184242    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184244    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184247    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184249    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184253    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184255    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184259    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184261    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184264    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184267    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184269    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184675    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184679    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184683    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184686    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184689    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:17:24.427276 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184691    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184696    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184699    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184702    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184705    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184708    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184711    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184714    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184716    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184721    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184724    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184727    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184729    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184732    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184735    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184738    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184740    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184743    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184746    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184748    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:17:24.429438 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184752    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184754    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184757    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184759    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184762    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184765    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184768    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184770    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184773    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184775    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184778    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184780    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184783    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184785    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184788    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184790    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184793    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184795    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184798    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184800    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:17:24.430747 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184803    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184807    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184811    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184813    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184816    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184819    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184822    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184825    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184828    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184830    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184832    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184835    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184838    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184841    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184844    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184846    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184848    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184851    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184854    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:17:24.431600 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184856    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184859    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184863    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184866    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184869    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184871    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184874    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184876    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184878    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184883    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184886    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184888    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184891    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184896    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184901    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184904    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184907    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184910    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184913    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:17:24.435049 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184915    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184918    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184921    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184923    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184926    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184928    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184931    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184933    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184936    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184939    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184942    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184944    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184946    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184949    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184951    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.184954    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185021    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185028    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185033    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185038    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185042    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 22:17:24.435902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185045    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185051    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185056    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185059    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185062    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185065    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185071    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185075    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185078    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185081    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185084    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185087    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185090    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185095    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185098    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185101    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185104    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185107    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185111    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185114    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185118    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185121    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185124    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185127    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 22:17:24.438036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185145    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185148    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185151    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185156    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185159    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185162    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185164    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185167    2564 flags.go:64] FLAG: --enable-server="true"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185170    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185176    2564 flags.go:64] FLAG: --event-burst="100"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185179    2564 flags.go:64] FLAG: --event-qps="50"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185182    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185185    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185188    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185193    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185196    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185199    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185202    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185205    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185208    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185211    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185213    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185216    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185219    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185222    2564 flags.go:64] FLAG: --feature-gates=""
Jul 16 22:17:24.438908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185226    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185228    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185232    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185235    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185238    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185241    2564 flags.go:64] FLAG: --help="false"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185244    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-136-7.ec2.internal"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185263    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185267    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185270    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185273    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185276    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185279    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185282    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185285    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185288    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185292    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185295    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185298    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185301    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185304    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185308    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185311    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185314    2564 flags.go:64] FLAG: --lock-file=""
Jul 16 22:17:24.441084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185316    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185319    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185322    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185328    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185331    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185334    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185336    2564 flags.go:64] FLAG: --logging-format="text"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185339    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185342    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185345    2564 flags.go:64] FLAG: --manifest-url=""
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185348    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185352    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185355    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185360    2564 flags.go:64] FLAG: --max-pods="110"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185364    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185366    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185369    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185372    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185375    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185378    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185381    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185387    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185390    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185393    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 22:17:24.441951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185397    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185400    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185406    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185409    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185412    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185415    2564 flags.go:64] FLAG: --port="10250"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185420    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185423    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03d77e13baecf3e83"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185426    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185429    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185432    2564 flags.go:64] FLAG: --register-node="true"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185435    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185438    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185441    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185444    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185446    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185451    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185454    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185456    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185460    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185462    2564 flags.go:64] FLAG: --runonce="false"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185465    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185469    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185472    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185475    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 22:17:24.442808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185477    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185480    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185483    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185486    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185489    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185492    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185495    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185499    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185502    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185506    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185508    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185513    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185516    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185520    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185525    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185527    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185530    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185533    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185536    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185539    2564 flags.go:64] FLAG: --v="2"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185543    2564 flags.go:64] FLAG: --version="false"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185547    2564 flags.go:64] FLAG: --vmodule=""
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185550    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.185553    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185669    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:17:24.443649 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185673    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185677    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185679    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185682    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185685    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185689    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185691    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185694    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185697    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185700    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185702    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185705    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185707    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185710    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185714    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185716    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185719    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185722    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185724    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185727    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:17:24.444518 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185730    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185733    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185736    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185738    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185741    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185744    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185746    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185749    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185753    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185757    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185760    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185762    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185765    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185767    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185770    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185772    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185775    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185778    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185785    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185788    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:17:24.445388 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185790    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185793    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185795    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185798    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185800    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185803    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185807    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185809    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185812    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185814    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185817    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185820    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185823    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185826    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185828    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185831    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185833    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185836    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185839    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185841    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:17:24.446679 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185844    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185847    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185849    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185851    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185854    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185857    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185859    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185862    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185864    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185868    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185872    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185876    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185879    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185882    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185885    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185887    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185890    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185892    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185896    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:17:24.447505 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185899    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185901    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185904    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185906    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185909    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185913    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185915    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185918    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185921    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185923    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185926    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185928    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185930    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185933    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185936    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185938    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185940    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185944    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:17:24.448805 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.185948    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.186433    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.193164    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.193178    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.193196    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.193203    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193261    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193266    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193270    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193274    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193278    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193281    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193284    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193288    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193292    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:17:24.449664 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193295    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193298    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193301    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193303    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193306    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193309    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193312    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193315    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193317    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193320    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193323    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193325    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193328    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193330    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193333    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193336    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193339    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193342    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193344    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193347    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:17:24.450102 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193349    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193352    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193355    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193357    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193360    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193363    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193366    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193369    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193371    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193373    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193376    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193378    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193381    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193384    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193386    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193389    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193391    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193394    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193397    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193399    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:17:24.450962 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193402    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193404    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193407    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193409    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193412    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193415    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193417    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193420    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193424    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193427    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193429    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193432    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193435    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193438    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193440    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193443    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193445    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193448    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193450    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193453    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:17:24.453126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193455    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193458    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193461    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193463    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193466    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193469    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193471    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193474    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193477    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193480    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193482    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193485    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193487    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193490    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193492    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193495    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193498    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193500    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193504    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:17:24.453986 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193508    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193512    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193515    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193518    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193521    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193524    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193526    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193529    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193532    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193534    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193539    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.193544    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193657    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193663    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:17:24.457460 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193666    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193669    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193672    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193674    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193678    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193682    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193685    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193689    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193691    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193694    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193696    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193699    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193702    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193704    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193708    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193711    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193714    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193717    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193720    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193723    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:17:24.457889 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193729    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193731    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193734    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193737    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193739    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193742    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193744    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193747    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193750    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193752    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193755    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193758    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193760    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193763    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193765    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193768    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193770    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193773    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193776    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193779    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:17:24.560742 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193782    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193784    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193787    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193789    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193792    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193794    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193796    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193799    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193801    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193804    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193807    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193809    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193814    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193816    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193819    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193821    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193824    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193826    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193829    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:17:24.561539 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193831    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193834    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193836    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193839    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193842    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193844    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193847    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193849    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193852    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193854    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193857    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193859    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193863    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193866    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193868    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193872    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193874    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193876    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193879    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193881    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:17:24.562349 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193884    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193886    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193889    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193892    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193894    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193898    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193900    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193903    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193906    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193908    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193912    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193915    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193918    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193921    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193924    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193926    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193929    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:17:24.563182 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:24.193932    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.193936    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.194040    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.197488    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.199112    2564 server.go:1019] "Starting client certificate rotation"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.199234    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.199259    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.201434    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.206545    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.226755    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.227040    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.230018    2564 log.go:25] "Validated CRI v1 image API"
Jul 16 22:17:24.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.230855    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 22:17:24.565593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.233247    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b197cff3-1195-4b39-9a32-21e0a9e84440:/dev/nvme0n1p4 c530a8ae-858c-4cfb-aafb-36594f2f6a88:/dev/nvme0n1p3]
Jul 16 22:17:24.565593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.233262    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.239905    2564 manager.go:216] Machine: {Timestamp:2026-07-16 22:17:24.239381044 +0000 UTC m=+0.559937112 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100770 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2830e70a1287f53c787d3117746b50 SystemUUID:ec2830e7-0a12-87f5-3c78-7d3117746b50 BootID:56f660ab-f2ea-4285-b52e-dc1827120fdc Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:62:23:7f:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:62:23:7f:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:ff:44:06:cb:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.240047    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.240195    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.242855    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.242883    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-7.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.243531    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.243540    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.243549    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.244114    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.245781    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249051    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249198    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249224    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249240    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249250    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249259    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249268    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 22:17:24.566852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249619    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.249642    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.253490    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6jnxk"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.253661    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.255408    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.255435    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256048    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256060    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256067    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256073    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256082    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256088    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256096    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256102    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256109    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256114    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256123    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256128    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256937    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.256945    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.257755    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6jnxk"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.260078    2564 server.go:1268] "Started kubelet"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.260522    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.260599    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.260941    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.262315    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-7.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.262473    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.262563    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-7.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.262824    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.265252    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.269958    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.271627    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.271632    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.271753    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.272253    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.272270    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.272256    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.272631    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.272644    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273520    2564 factory.go:55] Registering systemd factory
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273538    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273771    2564 factory.go:153] Registering CRI-O factory
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273782    2564 factory.go:223] Registration of the crio container factory successfully
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273840    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273858    2564 factory.go:103] Registering Raw factory
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.273901    2564 manager.go:1196] Started watching for new ooms in manager
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.274061    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:17:24.567667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.274337    2564 manager.go:349] Starting recovery of all containers
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.277567    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-7.ec2.internal\" not found" node="ip-10-0-136-7.ec2.internal"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.286052    2564 manager.go:354] Recovery completed
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.290333    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292123    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292168    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292184    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292627    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292636    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292651    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.292653    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.293341    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.293365    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.293373    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.293760    2564 policy_none.go:49] "None policy: Start"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.293772    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.293782    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:24.294966    2564 policy_none.go:47] "Start"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.372274    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:24.570082 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.472576    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:24.573752 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.572874    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:24.681383 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.673238    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:24.801610 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.773824    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:24.922845 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.874235    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.047214 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:24.974602    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.167595 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.075023    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.179331 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.175403    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.280887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.198628    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 22:17:25.280887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.198822    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 22:17:25.280887 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.198836    2564 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://ab5a7624716964b908d949ca870eb2cc-5cbf0c8e93c16c0d.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events\": read tcp 10.0.136.7:48764->3.208.227.248:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-136-7.ec2.internal.18c2e50c82fd6523  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-7.ec2.internal,UID:ip-10-0-136-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-7.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-7.ec2.internal,},FirstTimestamp:2026-07-16 22:17:24.292150563 +0000 UTC m=+0.612706626,LastTimestamp:2026-07-16 22:17:24.292150563 +0000 UTC m=+0.612706626,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-7.ec2.internal,}"
Jul 16 22:17:25.280887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.261174    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 22:12:24 +0000 UTC" deadline="2028-02-12 20:40:21.57134032 +0000 UTC"
Jul 16 22:17:25.280887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.261203    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13822h22m56.310140233s"
Jul 16 22:17:25.280887 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.276173    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.389226 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.376589    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.432994 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.432968    2564 manager.go:337] "Starting Device Plugin manager"
Jul 16 22:17:25.433123 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.433010    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 22:17:25.433123 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.433023    2564 server.go:87] "Starting device plugin registration server"
Jul 16 22:17:25.433332 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.433319    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 22:17:25.433399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.433334    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 22:17:25.433619 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.433409    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 22:17:25.433718 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.433682    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 22:17:25.433718 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.433691    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 22:17:25.434045 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.434020    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 22:17:25.434097 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.434082    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-7.ec2.internal\" not found"
Jul 16 22:17:25.437316 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.437298    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-7.ec2.internal" not found
Jul 16 22:17:25.438067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.438055    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:17:25.453658 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.453630    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-7.ec2.internal" not found
Jul 16 22:17:25.513641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.513612    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-7.ec2.internal" not found
Jul 16 22:17:25.533835 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.533748    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:17:25.534831 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.534807    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:17:25.534925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.534840    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:17:25.534925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.534851    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:17:25.534925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.534881    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.543300 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.543279    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.594281 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.594232    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 22:17:25.595439 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.595425    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 22:17:25.595564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.595459    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 22:17:25.595564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.595480    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 22:17:25.595564 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:25.595512    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 22:17:25.600273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.600251    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:17:25.670433 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.670403    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:17:25.697343 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.695555    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"]
Jul 16 22:17:25.699012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.698995    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.699949 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.699935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.726869 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.726844    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.731158 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.731130    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.745093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.745065    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 22:17:25.745216 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.745153    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 22:17:25.780364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.780328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27fdd382c58bf631634221549cb3cf12-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"27fdd382c58bf631634221549cb3cf12\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.780364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.780366    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27fdd382c58bf631634221549cb3cf12-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"27fdd382c58bf631634221549cb3cf12\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.780554 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.780391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8daa969aab0eee12d0aa31e5b2b8339-config\") pod \"kube-apiserver-proxy-ip-10-0-136-7.ec2.internal\" (UID: \"a8daa969aab0eee12d0aa31e5b2b8339\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.881502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.881430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27fdd382c58bf631634221549cb3cf12-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"27fdd382c58bf631634221549cb3cf12\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.881502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.881477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8daa969aab0eee12d0aa31e5b2b8339-config\") pod \"kube-apiserver-proxy-ip-10-0-136-7.ec2.internal\" (UID: \"a8daa969aab0eee12d0aa31e5b2b8339\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.881502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.881502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27fdd382c58bf631634221549cb3cf12-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"27fdd382c58bf631634221549cb3cf12\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.881683 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.881442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27fdd382c58bf631634221549cb3cf12-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"27fdd382c58bf631634221549cb3cf12\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.881683 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.881539    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27fdd382c58bf631634221549cb3cf12-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"27fdd382c58bf631634221549cb3cf12\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:25.881683 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:25.881579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8daa969aab0eee12d0aa31e5b2b8339-config\") pod \"kube-apiserver-proxy-ip-10-0-136-7.ec2.internal\" (UID: \"a8daa969aab0eee12d0aa31e5b2b8339\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:26.051845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.051803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:26.051845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.051855    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"
Jul 16 22:17:26.252588 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.252500    2564 apiserver.go:52] "Watching apiserver"
Jul 16 22:17:26.258882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.258853    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 22:17:26.259270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.259244    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz","openshift-dns/node-resolver-2t5nh","openshift-image-registry/node-ca-h6tnf","openshift-multus/multus-kf5qd","openshift-multus/network-metrics-daemon-pkgx8","openshift-network-diagnostics/network-check-target-sswr2","openshift-network-operator/iptables-alerter-445dz","kube-system/global-pull-secret-syncer-vxgdd","kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6wdd2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal","openshift-multus/multus-additional-cni-plugins-kppsg","openshift-ovn-kubernetes/ovnkube-node-2qxqw","kube-system/konnectivity-agent-j27xc"]
Jul 16 22:17:26.260931 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.260916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.262466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.262446    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.263340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.263324    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lqk6p\""
Jul 16 22:17:26.263430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.263390    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 22:17:26.263430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.263414    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.263677 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.263660    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.263746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.263694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.264480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.264443    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.264607 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.264592    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxtpw\""
Jul 16 22:17:26.264662 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.264621    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.265370 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.265353    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.265761 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.265745    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.265832 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.265769    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.265880 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.265750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 22:17:26.265880 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.265843    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j22mb\""
Jul 16 22:17:26.266950 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.266933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.267031 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.267002    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:26.267251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.267235    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 22:17:26.267523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.267483    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jrp5w\""
Jul 16 22:17:26.268636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.268005    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 22:17:26.268636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.268458    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.268838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.268693    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.270205 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.270186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:26.270281 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.270248    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:26.271432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.271412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.272031 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.272016    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 22:17:26.273078 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.273065    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.273158 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.273113    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:26.273762 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.273749    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gk8jc\""
Jul 16 22:17:26.273807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.273755    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 22:17:26.273807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.273768    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.273865 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.273761    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.274695 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.274678    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.276642 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.276624    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.276642 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.276637    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ds6kb\""
Jul 16 22:17:26.276875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.276861    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.276924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.276905    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.278935 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.278919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.279479 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.279464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fj487\""
Jul 16 22:17:26.279706 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.279689    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 22:17:26.279783 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.279732    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 22:17:26.280455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.280441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.280971 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.280953    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 22:17:26.281049 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.281005    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 22:17:26.281291 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.281274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 22:17:26.281377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.281296    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 22:17:26.281377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.281303    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 22:17:26.281520 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.281507    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jd654\""
Jul 16 22:17:26.282407 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282394    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 22:17:26.282453 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-sys-fs\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.282453 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-hostroot\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cd31ef13-2d2f-45f0-83b1-407feef1f326-hosts-file\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.282541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282473    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px7xs\" (UniqueName: \"kubernetes.io/projected/cd31ef13-2d2f-45f0-83b1-407feef1f326-kube-api-access-px7xs\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.282541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282518    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:26.282668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4f5d6fc-625d-4e57-b574-d539cd5576fe-host-slash\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.282668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-kubelet\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ec7c34d-4960-4611-8f14-9f5dac925696-kubelet-config\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.282668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-system-cni-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282654    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-socket-dir-parent\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-cni-bin\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282687    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-multus-certs\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd31ef13-2d2f-45f0-83b1-407feef1f326-tmp-dir\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-socket-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-cnibin\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-os-release\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282830    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-k8s-cni-cncf-io\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b4f5d6fc-625d-4e57-b574-d539cd5576fe-iptables-alerter-script\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.282860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl7wr\" (UniqueName: \"kubernetes.io/projected/b4f5d6fc-625d-4e57-b574-d539cd5576fe-kube-api-access-sl7wr\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6wvd\" (UniqueName: \"kubernetes.io/projected/158c8f12-045b-4e96-b870-e71eb26d1365-kube-api-access-j6wvd\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282919    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfmzt\" (UniqueName: \"kubernetes.io/projected/090b782f-d7ad-454f-bda9-93f33dd31d61-kube-api-access-jfmzt\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/158c8f12-045b-4e96-b870-e71eb26d1365-serviceca\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-cni-multus\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.282987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b1e5304-a728-4a7d-8627-e0f909844590-multus-daemon-config\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283002    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxx7x\" (UniqueName: \"kubernetes.io/projected/7b1e5304-a728-4a7d-8627-e0f909844590-kube-api-access-sxx7x\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-device-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-cni-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b1e5304-a728-4a7d-8627-e0f909844590-cni-binary-copy\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-etc-kubernetes\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc4sw\" (UniqueName: \"kubernetes.io/projected/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-kube-api-access-xc4sw\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283167    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ec7c34d-4960-4611-8f14-9f5dac925696-dbus\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283194    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-registration-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.283244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/158c8f12-045b-4e96-b870-e71eb26d1365-host\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.283672 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-netns\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.283672 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.283247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-conf-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.285026 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.285010    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 22:17:26.285186 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.285169    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 22:17:26.285254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.285235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gph8l\""
Jul 16 22:17:26.287210 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.287191    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 22:17:26.304848 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.304823    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f4nss"
Jul 16 22:17:26.312938 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.312917    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f4nss"
Jul 16 22:17:26.373244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.373227    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 22:17:26.383704 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-hostroot\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.383850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4f5d6fc-625d-4e57-b574-d539cd5576fe-host-slash\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.383850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-hostroot\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.383850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ec7c34d-4960-4611-8f14-9f5dac925696-kubelet-config\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.383850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-os-release\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.383850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-var-lib-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.383850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ec7c34d-4960-4611-8f14-9f5dac925696-kubelet-config\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4f5d6fc-625d-4e57-b574-d539cd5576fe-host-slash\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-systemd\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.383962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-system-cni-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-systemd-units\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-system-cni-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-slash\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-run-netns\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-systemd\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovn-node-metrics-cert\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-modprobe-d\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-multus-certs\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd31ef13-2d2f-45f0-83b1-407feef1f326-tmp-dir\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysctl-d\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384429    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-etc-selinux\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384484    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysctl-conf\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384537    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-multus-certs\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-socket-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-os-release\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.384656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl7wr\" (UniqueName: \"kubernetes.io/projected/b4f5d6fc-625d-4e57-b574-d539cd5576fe-kube-api-access-sl7wr\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd31ef13-2d2f-45f0-83b1-407feef1f326-tmp-dir\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384686    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovnkube-config\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-host\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-os-release\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384873    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b1e5304-a728-4a7d-8627-e0f909844590-multus-daemon-config\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384884    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-socket-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.384907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc4sw\" (UniqueName: \"kubernetes.io/projected/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-kube-api-access-xc4sw\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ec7c34d-4960-4611-8f14-9f5dac925696-dbus\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-kubernetes\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385083    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-device-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b1e5304-a728-4a7d-8627-e0f909844590-cni-binary-copy\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385146    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-lib-modules\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-registration-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.385423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/158c8f12-045b-4e96-b870-e71eb26d1365-host\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-netns\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1aafd55-4f83-4130-834f-0c70c8267793-konnectivity-ca\") pod \"konnectivity-agent-j27xc\" (UID: \"f1aafd55-4f83-4130-834f-0c70c8267793\") " pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-sys-fs\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cd31ef13-2d2f-45f0-83b1-407feef1f326-hosts-file\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-netns\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px7xs\" (UniqueName: \"kubernetes.io/projected/cd31ef13-2d2f-45f0-83b1-407feef1f326-kube-api-access-px7xs\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385434    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-sys-fs\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385432    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b1e5304-a728-4a7d-8627-e0f909844590-multus-daemon-config\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ec7c34d-4960-4611-8f14-9f5dac925696-dbus\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-registration-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/090b782f-d7ad-454f-bda9-93f33dd31d61-device-dir\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cd31ef13-2d2f-45f0-83b1-407feef1f326-hosts-file\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/158c8f12-045b-4e96-b870-e71eb26d1365-host\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-log-socket\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385582    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b1e5304-a728-4a7d-8627-e0f909844590-cni-binary-copy\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysconfig\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385682    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-run\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-kubelet\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnh29\" (UniqueName: \"kubernetes.io/projected/66be930b-284c-4b58-a820-c0fdf15fe882-kube-api-access-gnh29\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-kubelet\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-socket-dir-parent\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-socket-dir-parent\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-cni-bin\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-cni-bin\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-system-cni-dir\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-cnibin\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.385907    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-node-log\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.385963    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:26.885936402 +0000 UTC m=+3.206492457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385976    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-cni-netd\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.385993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66be930b-284c-4b58-a820-c0fdf15fe882-tmp\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.386777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-kubelet\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-cnibin\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-k8s-cni-cncf-io\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b4f5d6fc-625d-4e57-b574-d539cd5576fe-iptables-alerter-script\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-cnibin\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-run-k8s-cni-cncf-io\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-env-overrides\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386128    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovnkube-script-lib\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6wvd\" (UniqueName: \"kubernetes.io/projected/158c8f12-045b-4e96-b870-e71eb26d1365-kube-api-access-j6wvd\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfmzt\" (UniqueName: \"kubernetes.io/projected/090b782f-d7ad-454f-bda9-93f33dd31d61-kube-api-access-jfmzt\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.386187    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/158c8f12-045b-4e96-b870-e71eb26d1365-serviceca\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-cni-multus\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.386237    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:26.886221898 +0000 UTC m=+3.206777948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386247    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-host-var-lib-cni-multus\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx7x\" (UniqueName: \"kubernetes.io/projected/7b1e5304-a728-4a7d-8627-e0f909844590-kube-api-access-sxx7x\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-cni-bin\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1aafd55-4f83-4130-834f-0c70c8267793-agent-certs\") pod \"konnectivity-agent-j27xc\" (UID: \"f1aafd55-4f83-4130-834f-0c70c8267793\") " pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-cni-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-etc-kubernetes\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-ovn\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-sys\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-etc-kubernetes\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-conf-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-cni-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9v7g\" (UniqueName: \"kubernetes.io/projected/f584b484-08cd-4488-8683-954e160667c4-kube-api-access-d9v7g\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b4f5d6fc-625d-4e57-b574-d539cd5576fe-iptables-alerter-script\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386591    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-etc-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wlk8\" (UniqueName: \"kubernetes.io/projected/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-kube-api-access-8wlk8\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386622    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-var-lib-kubelet\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.387858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b1e5304-a728-4a7d-8627-e0f909844590-multus-conf-dir\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.388342 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/158c8f12-045b-4e96-b870-e71eb26d1365-serviceca\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.388342 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.386644    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66be930b-284c-4b58-a820-c0fdf15fe882-etc-tuned\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.390461 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.390440    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 22:17:26.391128 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.391111    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:26.391128 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.391128    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:26.391284 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.391153    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:26.391284 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.391195    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:26.891184432 +0000 UTC m=+3.211740483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:26.393708 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.393688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc4sw\" (UniqueName: \"kubernetes.io/projected/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-kube-api-access-xc4sw\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.393708 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.393700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl7wr\" (UniqueName: \"kubernetes.io/projected/b4f5d6fc-625d-4e57-b574-d539cd5576fe-kube-api-access-sl7wr\") pod \"iptables-alerter-445dz\" (UID: \"b4f5d6fc-625d-4e57-b574-d539cd5576fe\") " pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.393851 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.393740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px7xs\" (UniqueName: \"kubernetes.io/projected/cd31ef13-2d2f-45f0-83b1-407feef1f326-kube-api-access-px7xs\") pod \"node-resolver-2t5nh\" (UID: \"cd31ef13-2d2f-45f0-83b1-407feef1f326\") " pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.396493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.396474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxx7x\" (UniqueName: \"kubernetes.io/projected/7b1e5304-a728-4a7d-8627-e0f909844590-kube-api-access-sxx7x\") pod \"multus-kf5qd\" (UID: \"7b1e5304-a728-4a7d-8627-e0f909844590\") " pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.396493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.396484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6wvd\" (UniqueName: \"kubernetes.io/projected/158c8f12-045b-4e96-b870-e71eb26d1365-kube-api-access-j6wvd\") pod \"node-ca-h6tnf\" (UID: \"158c8f12-045b-4e96-b870-e71eb26d1365\") " pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.397038 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.397022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfmzt\" (UniqueName: \"kubernetes.io/projected/090b782f-d7ad-454f-bda9-93f33dd31d61-kube-api-access-jfmzt\") pod \"aws-ebs-csi-driver-node-qwxhz\" (UID: \"090b782f-d7ad-454f-bda9-93f33dd31d61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.487630 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-env-overrides\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.487630 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovnkube-script-lib\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-cni-bin\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1aafd55-4f83-4130-834f-0c70c8267793-agent-certs\") pod \"konnectivity-agent-j27xc\" (UID: \"f1aafd55-4f83-4130-834f-0c70c8267793\") " pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-cni-bin\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-ovn\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-sys\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9v7g\" (UniqueName: \"kubernetes.io/projected/f584b484-08cd-4488-8683-954e160667c4-kube-api-access-d9v7g\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-etc-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.487887 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wlk8\" (UniqueName: \"kubernetes.io/projected/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-kube-api-access-8wlk8\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-var-lib-kubelet\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-ovn\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66be930b-284c-4b58-a820-c0fdf15fe882-etc-tuned\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.487965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-os-release\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-var-lib-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-systemd\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-systemd-units\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-slash\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-run-netns\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-systemd\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovn-node-metrics-cert\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-modprobe-d\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-env-overrides\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.488381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysctl-d\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488322    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-slash\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488325    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488340    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovnkube-script-lib\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-systemd\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-sys\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-systemd\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-etc-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysctl-conf\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488455    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-var-lib-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-run-openvswitch\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovnkube-config\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488554    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-host\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-kubernetes\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-lib-modules\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488626    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-systemd-units\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1aafd55-4f83-4130-834f-0c70c8267793-konnectivity-ca\") pod \"konnectivity-agent-j27xc\" (UID: \"f1aafd55-4f83-4130-834f-0c70c8267793\") " pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488651    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-modprobe-d\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-log-socket\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysconfig\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-log-socket\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysctl-conf\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488784    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysconfig\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488835    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-kubernetes\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-var-lib-kubelet\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488874    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-host\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f584b484-08cd-4488-8683-954e160667c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-etc-sysctl-d\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.488962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-lib-modules\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-os-release\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.489914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-run\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnh29\" (UniqueName: \"kubernetes.io/projected/66be930b-284c-4b58-a820-c0fdf15fe882-kube-api-access-gnh29\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-system-cni-dir\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-cnibin\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-run-netns\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66be930b-284c-4b58-a820-c0fdf15fe882-run\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-node-log\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-cni-netd\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovnkube-config\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66be930b-284c-4b58-a820-c0fdf15fe882-tmp\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-node-log\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-cnibin\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f584b484-08cd-4488-8683-954e160667c4-system-cni-dir\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-kubelet\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-cni-netd\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-kubelet\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.490553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.491087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.489965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1aafd55-4f83-4130-834f-0c70c8267793-konnectivity-ca\") pod \"konnectivity-agent-j27xc\" (UID: \"f1aafd55-4f83-4130-834f-0c70c8267793\") " pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.491087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.490399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66be930b-284c-4b58-a820-c0fdf15fe882-etc-tuned\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.491174 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.491149    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-ovn-node-metrics-cert\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.491393 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.491371    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1aafd55-4f83-4130-834f-0c70c8267793-agent-certs\") pod \"konnectivity-agent-j27xc\" (UID: \"f1aafd55-4f83-4130-834f-0c70c8267793\") " pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.491434 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.491421    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66be930b-284c-4b58-a820-c0fdf15fe882-tmp\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.496295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.496271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9v7g\" (UniqueName: \"kubernetes.io/projected/f584b484-08cd-4488-8683-954e160667c4-kube-api-access-d9v7g\") pod \"multus-additional-cni-plugins-kppsg\" (UID: \"f584b484-08cd-4488-8683-954e160667c4\") " pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.496447 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.496428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wlk8\" (UniqueName: \"kubernetes.io/projected/d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef-kube-api-access-8wlk8\") pod \"ovnkube-node-2qxqw\" (UID: \"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.497276 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.497261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnh29\" (UniqueName: \"kubernetes.io/projected/66be930b-284c-4b58-a820-c0fdf15fe882-kube-api-access-gnh29\") pod \"tuned-6wdd2\" (UID: \"66be930b-284c-4b58-a820-c0fdf15fe882\") " pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.597472 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.597398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz"
Jul 16 22:17:26.618185 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.618157    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2t5nh"
Jul 16 22:17:26.632957 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.632936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6tnf"
Jul 16 22:17:26.641658 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.641618    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158c8f12_045b_4e96_b870_e71eb26d1365.slice/crio-446a607428dc6430844ad3c2272293188b9c710a00b996ed4e47208d1c1137cb WatchSource:0}: Error finding container 446a607428dc6430844ad3c2272293188b9c710a00b996ed4e47208d1c1137cb: Status 404 returned error can't find the container with id 446a607428dc6430844ad3c2272293188b9c710a00b996ed4e47208d1c1137cb
Jul 16 22:17:26.642319 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.642296    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fdd382c58bf631634221549cb3cf12.slice/crio-35adf919a0e98930d6c3ee82831a1623e01b5074012aebfd8fc4e48fc10ee4ac WatchSource:0}: Error finding container 35adf919a0e98930d6c3ee82831a1623e01b5074012aebfd8fc4e48fc10ee4ac: Status 404 returned error can't find the container with id 35adf919a0e98930d6c3ee82831a1623e01b5074012aebfd8fc4e48fc10ee4ac
Jul 16 22:17:26.643313 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.643295    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd31ef13_2d2f_45f0_83b1_407feef1f326.slice/crio-d4c719c85e7b5226020d70afacbff064806ba4c0b62f893f3186f2cec9ef1a06 WatchSource:0}: Error finding container d4c719c85e7b5226020d70afacbff064806ba4c0b62f893f3186f2cec9ef1a06: Status 404 returned error can't find the container with id d4c719c85e7b5226020d70afacbff064806ba4c0b62f893f3186f2cec9ef1a06
Jul 16 22:17:26.644035 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.644013    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8daa969aab0eee12d0aa31e5b2b8339.slice/crio-ca1c80998eb60451e025cb18efb542250c0f2d42fdee1db43077516ad07d7a93 WatchSource:0}: Error finding container ca1c80998eb60451e025cb18efb542250c0f2d42fdee1db43077516ad07d7a93: Status 404 returned error can't find the container with id ca1c80998eb60451e025cb18efb542250c0f2d42fdee1db43077516ad07d7a93
Jul 16 22:17:26.647937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.647921    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:17:26.651060 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.651041    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kf5qd"
Jul 16 22:17:26.659255 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.659233    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1e5304_a728_4a7d_8627_e0f909844590.slice/crio-ed645ab1aabe7d9560699d5a9b8a09d0d378de1f3419bcb714495cd60688d5cb WatchSource:0}: Error finding container ed645ab1aabe7d9560699d5a9b8a09d0d378de1f3419bcb714495cd60688d5cb: Status 404 returned error can't find the container with id ed645ab1aabe7d9560699d5a9b8a09d0d378de1f3419bcb714495cd60688d5cb
Jul 16 22:17:26.667166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.667127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-445dz"
Jul 16 22:17:26.674062 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.674037    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f5d6fc_625d_4e57_b574_d539cd5576fe.slice/crio-90978f630cba43e87567d8f93501852a1edd67ba21fca54aba2a036cdc91ef40 WatchSource:0}: Error finding container 90978f630cba43e87567d8f93501852a1edd67ba21fca54aba2a036cdc91ef40: Status 404 returned error can't find the container with id 90978f630cba43e87567d8f93501852a1edd67ba21fca54aba2a036cdc91ef40
Jul 16 22:17:26.684034 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.684015    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6wdd2"
Jul 16 22:17:26.690584 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.690561    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66be930b_284c_4b58_a820_c0fdf15fe882.slice/crio-44e04be9d7eaf72826d2d028941e2f85db5c7f0be9337b508e7a4fc2aae94ff9 WatchSource:0}: Error finding container 44e04be9d7eaf72826d2d028941e2f85db5c7f0be9337b508e7a4fc2aae94ff9: Status 404 returned error can't find the container with id 44e04be9d7eaf72826d2d028941e2f85db5c7f0be9337b508e7a4fc2aae94ff9
Jul 16 22:17:26.704006 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.703975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kppsg"
Jul 16 22:17:26.711350 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.711326    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf584b484_08cd_4488_8683_954e160667c4.slice/crio-7e559fac6602993f6611dd411359a615ae404f65c22272b3549db99a07747512 WatchSource:0}: Error finding container 7e559fac6602993f6611dd411359a615ae404f65c22272b3549db99a07747512: Status 404 returned error can't find the container with id 7e559fac6602993f6611dd411359a615ae404f65c22272b3549db99a07747512
Jul 16 22:17:26.722349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.722332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:26.725166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.725152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:26.730355 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.730332    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2fa47a0_2e48_42d4_a75d_a1c177d8b6ef.slice/crio-bcf93ba5477928cb1441778fa15fe971f4b34b8382cc07c0fa1ca406ab81b8f2 WatchSource:0}: Error finding container bcf93ba5477928cb1441778fa15fe971f4b34b8382cc07c0fa1ca406ab81b8f2: Status 404 returned error can't find the container with id bcf93ba5477928cb1441778fa15fe971f4b34b8382cc07c0fa1ca406ab81b8f2
Jul 16 22:17:26.733590 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:26.733569    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1aafd55_4f83_4130_834f_0c70c8267793.slice/crio-29b4fc2ad58c135be483546997e69f76ce90c892fd324dde6f57c4912504e31a WatchSource:0}: Error finding container 29b4fc2ad58c135be483546997e69f76ce90c892fd324dde6f57c4912504e31a: Status 404 returned error can't find the container with id 29b4fc2ad58c135be483546997e69f76ce90c892fd324dde6f57c4912504e31a
Jul 16 22:17:26.763151 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.763112    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:17:26.892485 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.892397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:26.892485 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.892438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892536    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892559    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892579    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892589    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892592    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:27.892573997 +0000 UTC m=+4.213130048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:26.892563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892626    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892627    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:27.892614487 +0000 UTC m=+4.213170538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:26.892698 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:26.892677    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:27.892667581 +0000 UTC m=+4.213223632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:27.010705 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:17:27.010665    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090b782f_d7ad_454f_bda9_93f33dd31d61.slice/crio-32e1b80aa84a97cc0d0a0b2ada6906375c5a7be5a6b91bff5a6fdededeab3fa7 WatchSource:0}: Error finding container 32e1b80aa84a97cc0d0a0b2ada6906375c5a7be5a6b91bff5a6fdededeab3fa7: Status 404 returned error can't find the container with id 32e1b80aa84a97cc0d0a0b2ada6906375c5a7be5a6b91bff5a6fdededeab3fa7
Jul 16 22:17:27.315518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.315350    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 22:12:26 +0000 UTC" deadline="2028-01-06 23:28:58.616296633 +0000 UTC"
Jul 16 22:17:27.315518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.315412    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12937h11m31.300888229s"
Jul 16 22:17:27.597413 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.597287    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:27.597413 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.597316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:27.598351 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.597454    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:27.598351 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.597554    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:27.621578 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.621461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kf5qd" event={"ID":"7b1e5304-a728-4a7d-8627-e0f909844590","Type":"ContainerStarted","Data":"ed645ab1aabe7d9560699d5a9b8a09d0d378de1f3419bcb714495cd60688d5cb"}
Jul 16 22:17:27.627161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.627058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" event={"ID":"a8daa969aab0eee12d0aa31e5b2b8339","Type":"ContainerStarted","Data":"ca1c80998eb60451e025cb18efb542250c0f2d42fdee1db43077516ad07d7a93"}
Jul 16 22:17:27.645378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.645273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6tnf" event={"ID":"158c8f12-045b-4e96-b870-e71eb26d1365","Type":"ContainerStarted","Data":"446a607428dc6430844ad3c2272293188b9c710a00b996ed4e47208d1c1137cb"}
Jul 16 22:17:27.658593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.658561    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2t5nh" event={"ID":"cd31ef13-2d2f-45f0-83b1-407feef1f326","Type":"ContainerStarted","Data":"d4c719c85e7b5226020d70afacbff064806ba4c0b62f893f3186f2cec9ef1a06"}
Jul 16 22:17:27.669010 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.668971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" event={"ID":"27fdd382c58bf631634221549cb3cf12","Type":"ContainerStarted","Data":"35adf919a0e98930d6c3ee82831a1623e01b5074012aebfd8fc4e48fc10ee4ac"}
Jul 16 22:17:27.681738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.681698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz" event={"ID":"090b782f-d7ad-454f-bda9-93f33dd31d61","Type":"ContainerStarted","Data":"32e1b80aa84a97cc0d0a0b2ada6906375c5a7be5a6b91bff5a6fdededeab3fa7"}
Jul 16 22:17:27.688403 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.688369    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j27xc" event={"ID":"f1aafd55-4f83-4130-834f-0c70c8267793","Type":"ContainerStarted","Data":"29b4fc2ad58c135be483546997e69f76ce90c892fd324dde6f57c4912504e31a"}
Jul 16 22:17:27.695836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.695801    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"bcf93ba5477928cb1441778fa15fe971f4b34b8382cc07c0fa1ca406ab81b8f2"}
Jul 16 22:17:27.720081 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.720043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerStarted","Data":"7e559fac6602993f6611dd411359a615ae404f65c22272b3549db99a07747512"}
Jul 16 22:17:27.749911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.749868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6wdd2" event={"ID":"66be930b-284c-4b58-a820-c0fdf15fe882","Type":"ContainerStarted","Data":"44e04be9d7eaf72826d2d028941e2f85db5c7f0be9337b508e7a4fc2aae94ff9"}
Jul 16 22:17:27.763011 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.762963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-445dz" event={"ID":"b4f5d6fc-625d-4e57-b574-d539cd5576fe","Type":"ContainerStarted","Data":"90978f630cba43e87567d8f93501852a1edd67ba21fca54aba2a036cdc91ef40"}
Jul 16 22:17:27.901693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.901577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:27.901693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.901633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:27.901693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:27.901692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:27.902527 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.902120    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:27.902527 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.902177    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:27.902527 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.902195    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:27.902527 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.902274    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.902253309 +0000 UTC m=+6.222809363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:27.911317 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.911290    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:27.911451 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.911376    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.911354763 +0000 UTC m=+6.231910814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:27.911520 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.911461    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:27.911520 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:27.911507    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.91149427 +0000 UTC m=+6.232050323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:28.316527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:28.316464    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 22:12:26 +0000 UTC" deadline="2028-02-25 07:01:02.517278189 +0000 UTC"
Jul 16 22:17:28.316527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:28.316526    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14120h43m34.200756585s"
Jul 16 22:17:28.596690 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:28.596606    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:28.596854 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:28.596736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:29.597506 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:29.597278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:29.597506 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.597426    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:29.597506 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:29.597473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:29.598254 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.597616    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:29.920225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:29.920286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:29.920321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920454    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920515    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:33.920496899 +0000 UTC m=+10.241052956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920604    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920620    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920633    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920671    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:33.920659147 +0000 UTC m=+10.241215199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920730    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:29.920880 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:29.920760    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:33.920748453 +0000 UTC m=+10.241304506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:30.595735 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:30.595696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:30.595919 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:30.595842    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:31.597272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:31.597239    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:31.597713 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:31.597389    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:31.597850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:31.597826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:31.597971 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:31.597946    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:32.595932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:32.595898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:32.596121 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:32.596031    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:33.596294 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:33.596262    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:33.596758 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:33.596265    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:33.596758 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.596396    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:33.596758 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.596507    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:33.956179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:33.956237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:33.956271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956422    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956482    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:41.956464426 +0000 UTC m=+18.277020480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956611    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956657    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:41.956642731 +0000 UTC m=+18.277198786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956726    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956742    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956754    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:33.956874 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:33.956791    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:41.956779353 +0000 UTC m=+18.277335407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:34.596209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:34.596171    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:34.596413 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:34.596302    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:35.596870 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:35.596809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:35.597377 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:35.596939    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:35.597377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:35.596960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:35.597377 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:35.597062    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:36.596111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:36.596072    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:36.596314 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:36.596232    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:37.596805 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:37.596767    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:37.597280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:37.596772    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:37.597280 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:37.596935    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:37.597280 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:37.596990    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:38.596399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:38.596361    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:38.596582 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:38.596478    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:39.596174 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:39.596123    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:39.596632 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:39.596123    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:39.596632 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:39.596286    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:39.596632 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:39.596360    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:40.596474 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:40.596433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:40.596900 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:40.596557    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:41.595693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:41.595652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:41.595850 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:41.595783    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:41.595850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:41.595820    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:41.595967 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:41.595872    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:42.016663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:42.016570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:42.016663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:42.016628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:42.016759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016775    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016901    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016924    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016943    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016973    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:58.016951476 +0000 UTC m=+34.337507531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016848    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.016993    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:58.016982311 +0000 UTC m=+34.337538364 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:42.017110 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.017040    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:58.017002822 +0000 UTC m=+34.337558876 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:42.596218 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:42.596124    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:42.596375 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:42.596276    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:43.595787 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:43.595740    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:43.596179 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:43.595791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:43.596179 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:43.595892    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:43.596179 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:43.596006    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:44.596228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:44.596194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:44.596673 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:44.596313    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:45.597232 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:45.597188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:45.597686 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:45.597242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:45.597686 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:45.597327    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:45.597686 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:45.597441    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:46.596518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:46.596479    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:46.596683 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:46.596584    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:47.596091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.595902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:47.596499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.595919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:47.596499 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:47.596201    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:47.596499 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:47.596277    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:47.811749 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.811305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kf5qd" event={"ID":"7b1e5304-a728-4a7d-8627-e0f909844590","Type":"ContainerStarted","Data":"dbf43dd6902401ba08bdece328216d26ccaadf83ef932bae043bd890fb63e4e1"}
Jul 16 22:17:47.814308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.814226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" event={"ID":"a8daa969aab0eee12d0aa31e5b2b8339","Type":"ContainerStarted","Data":"0b81282dc7c33c1c5f17ce032aa5d962a35b01b5e5c00ac238b3c155900119f2"}
Jul 16 22:17:47.819233 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819215    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:17:47.819588 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819568    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef" containerID="22f55ce7196a814c8779938171e662de2a50f4c59e6333921b29bf742aca083d" exitCode=1
Jul 16 22:17:47.819662 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"fa653de26ef4d1eaaff0c6ddf76cccec00cf0abb00376d884da4bd3703270649"}
Jul 16 22:17:47.819728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"b263ecbd8477899fb69a448c872e840063a8ed5e187a490779d097fadbe83025"}
Jul 16 22:17:47.819780 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"37d617fadf3b3400d17b644098aa40a35e5f2e220b95d7dd2cd7bd1b7816c319"}
Jul 16 22:17:47.819832 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"030a8c78977eba4e043a4c5aac6fb990740813f3a2f9a4fbad6c82cf35fe15d4"}
Jul 16 22:17:47.819882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerDied","Data":"22f55ce7196a814c8779938171e662de2a50f4c59e6333921b29bf742aca083d"}
Jul 16 22:17:47.819921 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.819886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"42d1f45c2a98cbe47f3bf0b32c6c006e392fe6cf8f114e9f0839da9e01608b93"}
Jul 16 22:17:47.820977 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.820956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6wdd2" event={"ID":"66be930b-284c-4b58-a820-c0fdf15fe882","Type":"ContainerStarted","Data":"9c89e326ad895c9d7edb4586295ee0e902b9303308b3d238352b16336fde154e"}
Jul 16 22:17:47.831342 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.831300    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kf5qd" podStartSLOduration=2.709700361 podStartE2EDuration="22.831287449s" totalImagesPullingTime="20.121587088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:47.830834765 +0000 UTC m=+24.151390836" watchObservedRunningTime="2026-07-16 22:17:47.831287449 +0000 UTC m=+24.151843540"
Jul 16 22:17:47.847567 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.847516    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6wdd2" podStartSLOduration=2.789290333 podStartE2EDuration="22.847501164s" totalImagesPullingTime="20.058210831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:47.847051077 +0000 UTC m=+24.167607150" watchObservedRunningTime="2026-07-16 22:17:47.847501164 +0000 UTC m=+24.168057213"
Jul 16 22:17:47.861822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:47.861768    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" podStartSLOduration=22.861748269 podStartE2EDuration="22.861748269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:47.860900908 +0000 UTC m=+24.181456977" watchObservedRunningTime="2026-07-16 22:17:47.861748269 +0000 UTC m=+24.182304348"
Jul 16 22:17:48.596665 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.596420    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:48.597377 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:48.596688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:48.825024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.824983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz" event={"ID":"090b782f-d7ad-454f-bda9-93f33dd31d61","Type":"ContainerStarted","Data":"2207d79d789f473c8295599eac4f641b826af923d9ba783cc1f4630f791f8fc9"}
Jul 16 22:17:48.826705 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.826632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j27xc" event={"ID":"f1aafd55-4f83-4130-834f-0c70c8267793","Type":"ContainerStarted","Data":"23b02e535e15fb3ae1a33827792583627b7ed928b327049ccd9fcd1c0b893939"}
Jul 16 22:17:48.828266 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.828235    2564 generic.go:356] "Generic (PLEG): container finished" podID="f584b484-08cd-4488-8683-954e160667c4" containerID="4c0d04c43bed7fb5d6aff3821d5c1decc7dce05836f9ec7f6e3c3894ae2b57f5" exitCode=0
Jul 16 22:17:48.828398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.828328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerDied","Data":"4c0d04c43bed7fb5d6aff3821d5c1decc7dce05836f9ec7f6e3c3894ae2b57f5"}
Jul 16 22:17:48.830269 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.829969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-445dz" event={"ID":"b4f5d6fc-625d-4e57-b574-d539cd5576fe","Type":"ContainerStarted","Data":"8e0ff03e48ae16ca7b3a72f8bfff636fa9e9bf024b0739385ec173fa515370e8"}
Jul 16 22:17:48.831587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.831541    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6tnf" event={"ID":"158c8f12-045b-4e96-b870-e71eb26d1365","Type":"ContainerStarted","Data":"0599cc03542d98204ffe7d96d636fd125cbde4a832449422cb69272e67e817f1"}
Jul 16 22:17:48.833157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.833101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2t5nh" event={"ID":"cd31ef13-2d2f-45f0-83b1-407feef1f326","Type":"ContainerStarted","Data":"62cd2a2c1c572815b4b1bc9144d0f1b7539e89ce0ae9af0bad9b632f39de0107"}
Jul 16 22:17:48.834638 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.834616    2564 generic.go:356] "Generic (PLEG): container finished" podID="27fdd382c58bf631634221549cb3cf12" containerID="fb47ad3b3740a75e706f724774fc57a208d691e500fcbca9f9ec3e158408f8e0" exitCode=0
Jul 16 22:17:48.834742 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.834711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" event={"ID":"27fdd382c58bf631634221549cb3cf12","Type":"ContainerDied","Data":"fb47ad3b3740a75e706f724774fc57a208d691e500fcbca9f9ec3e158408f8e0"}
Jul 16 22:17:48.842637 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.842599    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j27xc" podStartSLOduration=4.168988406 podStartE2EDuration="23.842585788s" totalImagesPullingTime="19.673597382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:48.841894955 +0000 UTC m=+25.162451030" watchObservedRunningTime="2026-07-16 22:17:48.842585788 +0000 UTC m=+25.163141860"
Jul 16 22:17:48.870237 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.870014    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2t5nh" podStartSLOduration=3.76735876 podStartE2EDuration="23.869999795s" totalImagesPullingTime="20.102641035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:48.869842883 +0000 UTC m=+25.190398956" watchObservedRunningTime="2026-07-16 22:17:48.869999795 +0000 UTC m=+25.190555866"
Jul 16 22:17:48.906328 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.906280    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h6tnf" podStartSLOduration=3.836001752 podStartE2EDuration="23.906263468s" totalImagesPullingTime="20.070261716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:48.905695662 +0000 UTC m=+25.226251738" watchObservedRunningTime="2026-07-16 22:17:48.906263468 +0000 UTC m=+25.226819577"
Jul 16 22:17:48.922509 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:48.922460    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-445dz" podStartSLOduration=4.189422152 podStartE2EDuration="23.922443225s" totalImagesPullingTime="19.733021073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:48.922041101 +0000 UTC m=+25.242597174" watchObservedRunningTime="2026-07-16 22:17:48.922443225 +0000 UTC m=+25.242999297"
Jul 16 22:17:49.393836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.393809    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 22:17:49.449527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.449418    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T22:17:49.393830996Z","UUID":"65ea5e06-257e-48ee-bfac-3c7e2cc92097","Handler":null,"Name":"","Endpoint":""}
Jul 16 22:17:49.451403 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.451380    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 22:17:49.451529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.451426    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 22:17:49.598830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.598763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:49.599268 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:49.598865    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:49.599268 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.598917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:49.599268 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:49.599020    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:49.839354 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.839316    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz" event={"ID":"090b782f-d7ad-454f-bda9-93f33dd31d61","Type":"ContainerStarted","Data":"42bd22dcfba6dde7d3a47a63978bbc5c851c8c0fe88581c1991897b6ae514140"}
Jul 16 22:17:49.841309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.841250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" event={"ID":"27fdd382c58bf631634221549cb3cf12","Type":"ContainerStarted","Data":"76a499b2442cca0118fac249d53e83a94eb6a452703a0383ee2d50a4fc471615"}
Jul 16 22:17:49.857244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:49.857161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" podStartSLOduration=24.857125469 podStartE2EDuration="24.857125469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:49.856269611 +0000 UTC m=+26.176825685" watchObservedRunningTime="2026-07-16 22:17:49.857125469 +0000 UTC m=+26.177681542"
Jul 16 22:17:50.596096 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:50.596058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:50.596301 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:50.596200    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:50.845737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:50.845641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz" event={"ID":"090b782f-d7ad-454f-bda9-93f33dd31d61","Type":"ContainerStarted","Data":"c71f832af6fda0a8794d59096cbc836787d594292fce4fb893ed3b6e39c475be"}
Jul 16 22:17:50.851876 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:50.851853    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:17:50.852339 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:50.852307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"7a8fe16ed191aad5f74be98a942f51ea43789e6aa1cea6e9a34542dc099258f3"}
Jul 16 22:17:50.867547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:50.867490    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qwxhz" podStartSLOduration=4.061160833 podStartE2EDuration="25.867469663s" totalImagesPullingTime="21.80630883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:50.866463187 +0000 UTC m=+27.187019259" watchObservedRunningTime="2026-07-16 22:17:50.867469663 +0000 UTC m=+27.188025739"
Jul 16 22:17:51.596656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:51.596620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:51.596656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:51.596649    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:51.596912 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:51.596767    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:51.596912 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:51.596861    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:52.596083 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.595903    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:52.596859 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:52.596195    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:52.861427 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.861355    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:17:52.861804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.861769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"2f66e75c72a035e9cc0db79a953c2ef7a01e5aba05f4b3203888b71240b47b0b"}
Jul 16 22:17:52.862179 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.862155    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:52.862292 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.862236    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:52.862292 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.862286    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:52.862405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.862238    2564 scope.go:117] "RemoveContainer" containerID="22f55ce7196a814c8779938171e662de2a50f4c59e6333921b29bf742aca083d"
Jul 16 22:17:52.879661 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.879628    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:52.879843 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:52.879780    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:17:53.248886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:53.248791    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:53.249876 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:53.249844    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:53.599666 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:53.599601    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:53.600293 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:53.599698    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:53.600293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:53.599777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:53.600293 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:53.599912    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:53.864145 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:53.864036    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:53.864644 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:53.864621    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j27xc"
Jul 16 22:17:54.595777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:54.595752    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:54.595911 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:54.595848    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:54.869898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:54.869674    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:17:54.870283 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:54.870200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" event={"ID":"d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef","Type":"ContainerStarted","Data":"91d66fac049a1fafbe01b86c1ee44cb9205a64e0abcf52449b8b1199c4bce03b"}
Jul 16 22:17:54.872020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:54.871985    2564 generic.go:356] "Generic (PLEG): container finished" podID="f584b484-08cd-4488-8683-954e160667c4" containerID="3075a6abc87c1f04569b449370abfe86d744c173967e8b72531dcd2a80afd9ea" exitCode=0
Jul 16 22:17:54.872160 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:54.872046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerDied","Data":"3075a6abc87c1f04569b449370abfe86d744c173967e8b72531dcd2a80afd9ea"}
Jul 16 22:17:54.903449 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:54.903400    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw" podStartSLOduration=9.572659289 podStartE2EDuration="29.903384617s" totalImagesPullingTime="20.330725328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:54.90235482 +0000 UTC m=+31.222910893" watchObservedRunningTime="2026-07-16 22:17:54.903384617 +0000 UTC m=+31.223940688"
Jul 16 22:17:55.599704 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:55.599677    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:55.599866 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:55.599774    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:55.599866 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:55.599824    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:55.599951 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:55.599933    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:56.596560 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:56.596527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:56.597025 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:56.596649    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:56.878438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:56.878352    2564 generic.go:356] "Generic (PLEG): container finished" podID="f584b484-08cd-4488-8683-954e160667c4" containerID="9bc057271f3127972d294212bbe01b264407311bad692a77371229e1d9afd8da" exitCode=0
Jul 16 22:17:56.878438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:56.878396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerDied","Data":"9bc057271f3127972d294212bbe01b264407311bad692a77371229e1d9afd8da"}
Jul 16 22:17:57.596372 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:57.596344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:57.596559 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:57.596474    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:57.596559 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:57.596533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:57.596936 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:57.596636    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:17:58.033751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:58.033720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:58.033751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:58.033755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:58.033917 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:58.033775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:58.033917 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033866    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:58.033917 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033872    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:58.033917 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033894    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:58.033917 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033904    2564 projected.go:196] Error preparing data for projected volume kube-api-access-zknpf for pod openshift-network-diagnostics/network-check-target-sswr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:58.033917 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033911    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret podName:2ec7c34d-4960-4611-8f14-9f5dac925696 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:30.033898628 +0000 UTC m=+66.354454678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret") pod "global-pull-secret-syncer-vxgdd" (UID: "2ec7c34d-4960-4611-8f14-9f5dac925696") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:58.034108 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033941    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf podName:1b36dc0a-b970-442c-82cd-e35125a1bcb0 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:30.033930469 +0000 UTC m=+66.354486520 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zknpf" (UniqueName: "kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf") pod "network-check-target-sswr2" (UID: "1b36dc0a-b970-442c-82cd-e35125a1bcb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:58.034108 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033872    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:58.034108 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.033972    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs podName:68c5002e-be2b-48ff-9d4d-8a36d913f6b0 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:30.033965603 +0000 UTC m=+66.354521655 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs") pod "network-metrics-daemon-pkgx8" (UID: "68c5002e-be2b-48ff-9d4d-8a36d913f6b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:58.595714 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:58.595685    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:17:58.595870 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:58.595789    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:17:58.884499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:58.884414    2564 generic.go:356] "Generic (PLEG): container finished" podID="f584b484-08cd-4488-8683-954e160667c4" containerID="093985d9050073ffa0ed244070f0b324320997f5adcc6ac1ba1450643a489f19" exitCode=0
Jul 16 22:17:58.884499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:58.884464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerDied","Data":"093985d9050073ffa0ed244070f0b324320997f5adcc6ac1ba1450643a489f19"}
Jul 16 22:17:59.595913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:59.595881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:17:59.596107 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:59.596013    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:17:59.596107 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:17:59.596042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:17:59.596253 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:17:59.596165    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:18:00.595865 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:00.595837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:00.596293 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:00.595955    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:18:01.009188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:01.009128    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vxgdd"]
Jul 16 22:18:01.009377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:01.009276    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:01.009444 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:01.009386    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:18:01.012491 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:01.012125    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkgx8"]
Jul 16 22:18:01.012491 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:01.012236    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:01.012491 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:01.012332    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:18:01.013004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:01.012963    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sswr2"]
Jul 16 22:18:01.013219 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:01.013054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:01.013219 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:01.013215    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:18:02.596064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:02.596027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:02.596064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:02.596059    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:02.596532 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:02.596072    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:02.596532 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:02.596177    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:18:02.596532 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:02.596297    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:18:02.596532 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:02.596388    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:18:03.898786 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:03.898595    2564 generic.go:356] "Generic (PLEG): container finished" podID="f584b484-08cd-4488-8683-954e160667c4" containerID="643c434d1812a8d4bbb3fd3c017571749c57577cbd09821c5deab11ae6b3baa0" exitCode=0
Jul 16 22:18:03.898786 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:03.898673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerDied","Data":"643c434d1812a8d4bbb3fd3c017571749c57577cbd09821c5deab11ae6b3baa0"}
Jul 16 22:18:04.596320 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.596283    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:04.596486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.596288    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:04.596526 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.596299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:04.596526 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:04.596490    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sswr2" podUID="1b36dc0a-b970-442c-82cd-e35125a1bcb0"
Jul 16 22:18:04.596585 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:04.596402    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vxgdd" podUID="2ec7c34d-4960-4611-8f14-9f5dac925696"
Jul 16 22:18:04.596630 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:04.596609    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkgx8" podUID="68c5002e-be2b-48ff-9d4d-8a36d913f6b0"
Jul 16 22:18:04.903942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.903859    2564 generic.go:356] "Generic (PLEG): container finished" podID="f584b484-08cd-4488-8683-954e160667c4" containerID="fec26f2553abaaedb905b00e07f3638e827981b6d51564eb6b35a76944126122" exitCode=0
Jul 16 22:18:04.903942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.903928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerDied","Data":"fec26f2553abaaedb905b00e07f3638e827981b6d51564eb6b35a76944126122"}
Jul 16 22:18:04.914714 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.914693    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeReady"
Jul 16 22:18:04.914822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.914808    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 22:18:04.955060 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.955000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67968859f-8gm5q"]
Jul 16 22:18:04.986910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.986877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67968859f-8gm5q"]
Jul 16 22:18:04.987052 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.986924    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5l475"]
Jul 16 22:18:04.987052 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.987003    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:04.989541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.989517    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 22:18:04.990115 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.990100    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 22:18:04.990251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.990187    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 22:18:04.990308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.990291    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cmptb\""
Jul 16 22:18:04.995335 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:04.995317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 22:18:05.008954 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.008931    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5l475"]
Jul 16 22:18:05.009071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.009033    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.011855 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.011824    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 22:18:05.012006 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.011894    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4hsw2\""
Jul 16 22:18:05.012100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.012086    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 22:18:05.077867 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.077679    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q6g7l"]
Jul 16 22:18:05.086037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88f74aca-06f1-43f0-a26c-3daa3e07c667-installation-pull-secrets\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88f74aca-06f1-43f0-a26c-3daa3e07c667-image-registry-private-configuration\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88f74aca-06f1-43f0-a26c-3daa3e07c667-registry-certificates\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88f74aca-06f1-43f0-a26c-3daa3e07c667-ca-trust-extracted\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88f74aca-06f1-43f0-a26c-3daa3e07c667-trusted-ca\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-bound-sa-token\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086418 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxq9x\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-kube-api-access-cxq9x\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.086418 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.086340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-registry-tls\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.101120 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.101094    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tmz6v"]
Jul 16 22:18:05.101283 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.101216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.103487 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.103466    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 22:18:05.103608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.103543    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ssz9k\""
Jul 16 22:18:05.104062 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.104039    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 22:18:05.104159 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.104080    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 22:18:05.125194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.125155    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6g7l"]
Jul 16 22:18:05.125194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.125201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmz6v"]
Jul 16 22:18:05.125367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.125253    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.127503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.127480    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 22:18:05.127503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.127482    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 22:18:05.127681 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.127570    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cgkc2\""
Jul 16 22:18:05.127681 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.127575    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 22:18:05.127681 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.127658    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 22:18:05.186941 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.186851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88f74aca-06f1-43f0-a26c-3daa3e07c667-image-registry-private-configuration\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.186941 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.186894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88f74aca-06f1-43f0-a26c-3daa3e07c667-registry-certificates\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.186941 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.186922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zj9v\" (UniqueName: \"kubernetes.io/projected/7b8ce522-ef9d-4655-89a6-6e96b2fb0f12-kube-api-access-9zj9v\") pod \"ingress-canary-q6g7l\" (UID: \"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12\") " pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.186941 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.186946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88f74aca-06f1-43f0-a26c-3daa3e07c667-ca-trust-extracted\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.186969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-metrics-tls\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.186990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88f74aca-06f1-43f0-a26c-3daa3e07c667-trusted-ca\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-bound-sa-token\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq9x\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-kube-api-access-cxq9x\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-registry-tls\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b8ce522-ef9d-4655-89a6-6e96b2fb0f12-cert\") pod \"ingress-canary-q6g7l\" (UID: \"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12\") " pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-config-volume\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.187315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-tmp-dir\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.187836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88f74aca-06f1-43f0-a26c-3daa3e07c667-installation-pull-secrets\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzhfm\" (UniqueName: \"kubernetes.io/projected/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-kube-api-access-rzhfm\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.187836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88f74aca-06f1-43f0-a26c-3daa3e07c667-ca-trust-extracted\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.187836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88f74aca-06f1-43f0-a26c-3daa3e07c667-registry-certificates\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.188013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.187932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88f74aca-06f1-43f0-a26c-3daa3e07c667-trusted-ca\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.191255 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.191236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88f74aca-06f1-43f0-a26c-3daa3e07c667-installation-pull-secrets\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.191255 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.191249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88f74aca-06f1-43f0-a26c-3daa3e07c667-image-registry-private-configuration\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.191409 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.191389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-registry-tls\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.195073 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.195050    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxq9x\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-kube-api-access-cxq9x\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.195464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.195442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88f74aca-06f1-43f0-a26c-3daa3e07c667-bound-sa-token\") pod \"image-registry-67968859f-8gm5q\" (UID: \"88f74aca-06f1-43f0-a26c-3daa3e07c667\") " pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.288730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288693    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzhfm\" (UniqueName: \"kubernetes.io/projected/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-kube-api-access-rzhfm\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.288730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zj9v\" (UniqueName: \"kubernetes.io/projected/7b8ce522-ef9d-4655-89a6-6e96b2fb0f12-kube-api-access-9zj9v\") pod \"ingress-canary-q6g7l\" (UID: \"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12\") " pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.288973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-metrics-tls\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.288973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8e76aeb7-7a77-4efa-8798-780abcdf5801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.288973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b8ce522-ef9d-4655-89a6-6e96b2fb0f12-cert\") pod \"ingress-canary-q6g7l\" (UID: \"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12\") " pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.288973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh72f\" (UniqueName: \"kubernetes.io/projected/8e76aeb7-7a77-4efa-8798-780abcdf5801-kube-api-access-lh72f\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.288973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-config-volume\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.289228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.288981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-tmp-dir\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.289228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.289050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8e76aeb7-7a77-4efa-8798-780abcdf5801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.289228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.289096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8e76aeb7-7a77-4efa-8798-780abcdf5801-crio-socket\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.289228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.289126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e76aeb7-7a77-4efa-8798-780abcdf5801-data-volume\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.289400 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.289378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-tmp-dir\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.289529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.289511    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-config-volume\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.291209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.291190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-metrics-tls\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.291431 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.291414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b8ce522-ef9d-4655-89a6-6e96b2fb0f12-cert\") pod \"ingress-canary-q6g7l\" (UID: \"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12\") " pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.297257 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.297235    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.304579 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.304553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzhfm\" (UniqueName: \"kubernetes.io/projected/0a2a6174-b7bb-4201-82d5-8b4b59e930e1-kube-api-access-rzhfm\") pod \"dns-default-5l475\" (UID: \"0a2a6174-b7bb-4201-82d5-8b4b59e930e1\") " pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.307416 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.307390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zj9v\" (UniqueName: \"kubernetes.io/projected/7b8ce522-ef9d-4655-89a6-6e96b2fb0f12-kube-api-access-9zj9v\") pod \"ingress-canary-q6g7l\" (UID: \"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12\") " pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.319324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.319295    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:05.390305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh72f\" (UniqueName: \"kubernetes.io/projected/8e76aeb7-7a77-4efa-8798-780abcdf5801-kube-api-access-lh72f\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.390471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8e76aeb7-7a77-4efa-8798-780abcdf5801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.390471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8e76aeb7-7a77-4efa-8798-780abcdf5801-crio-socket\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.390471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e76aeb7-7a77-4efa-8798-780abcdf5801-data-volume\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.390616 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8e76aeb7-7a77-4efa-8798-780abcdf5801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.390685 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390655    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8e76aeb7-7a77-4efa-8798-780abcdf5801-crio-socket\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.390918 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.390894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e76aeb7-7a77-4efa-8798-780abcdf5801-data-volume\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.391176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.391116    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8e76aeb7-7a77-4efa-8798-780abcdf5801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.392642 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.392614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8e76aeb7-7a77-4efa-8798-780abcdf5801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.407659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.406337    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh72f\" (UniqueName: \"kubernetes.io/projected/8e76aeb7-7a77-4efa-8798-780abcdf5801-kube-api-access-lh72f\") pod \"insights-runtime-extractor-tmz6v\" (UID: \"8e76aeb7-7a77-4efa-8798-780abcdf5801\") " pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.410272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.410239    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6g7l"
Jul 16 22:18:05.437092 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.434308    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmz6v"
Jul 16 22:18:05.536492 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.536460    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5l475"]
Jul 16 22:18:05.538303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.538273    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67968859f-8gm5q"]
Jul 16 22:18:05.543274 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:05.542998    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2a6174_b7bb_4201_82d5_8b4b59e930e1.slice/crio-03bd0d833348023aedcdf5b6658c6d2e6cd80dd881075ec17e8e17631a779c3f WatchSource:0}: Error finding container 03bd0d833348023aedcdf5b6658c6d2e6cd80dd881075ec17e8e17631a779c3f: Status 404 returned error can't find the container with id 03bd0d833348023aedcdf5b6658c6d2e6cd80dd881075ec17e8e17631a779c3f
Jul 16 22:18:05.600256 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.600237    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6g7l"]
Jul 16 22:18:05.602084 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:05.602059    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8ce522_ef9d_4655_89a6_6e96b2fb0f12.slice/crio-cf92c242b7d0a9410f26c8177048c16a40d1178602d81e8bed6769087f7f6202 WatchSource:0}: Error finding container cf92c242b7d0a9410f26c8177048c16a40d1178602d81e8bed6769087f7f6202: Status 404 returned error can't find the container with id cf92c242b7d0a9410f26c8177048c16a40d1178602d81e8bed6769087f7f6202
Jul 16 22:18:05.612307 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.612281    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmz6v"]
Jul 16 22:18:05.615612 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:05.615585    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e76aeb7_7a77_4efa_8798_780abcdf5801.slice/crio-e4e95b1795779d13f925b57dff228f2f17eac05df96dc5b935a172b669f9e3fb WatchSource:0}: Error finding container e4e95b1795779d13f925b57dff228f2f17eac05df96dc5b935a172b669f9e3fb: Status 404 returned error can't find the container with id e4e95b1795779d13f925b57dff228f2f17eac05df96dc5b935a172b669f9e3fb
Jul 16 22:18:05.907878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.907840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5l475" event={"ID":"0a2a6174-b7bb-4201-82d5-8b4b59e930e1","Type":"ContainerStarted","Data":"03bd0d833348023aedcdf5b6658c6d2e6cd80dd881075ec17e8e17631a779c3f"}
Jul 16 22:18:05.908961 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.908937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6g7l" event={"ID":"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12","Type":"ContainerStarted","Data":"cf92c242b7d0a9410f26c8177048c16a40d1178602d81e8bed6769087f7f6202"}
Jul 16 22:18:05.910300 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.910276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67968859f-8gm5q" event={"ID":"88f74aca-06f1-43f0-a26c-3daa3e07c667","Type":"ContainerStarted","Data":"002fc89f1da1343c1b4fb30d16f24c1831653de3d07eba150ff539180ba1b6ff"}
Jul 16 22:18:05.910417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.910310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67968859f-8gm5q" event={"ID":"88f74aca-06f1-43f0-a26c-3daa3e07c667","Type":"ContainerStarted","Data":"50a83372fc378e7abdb7323c1af4e3838785694c10ead3fd7907109ae6fd7adc"}
Jul 16 22:18:05.910417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.910409    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:05.911530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.911508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmz6v" event={"ID":"8e76aeb7-7a77-4efa-8798-780abcdf5801","Type":"ContainerStarted","Data":"e4e95b1795779d13f925b57dff228f2f17eac05df96dc5b935a172b669f9e3fb"}
Jul 16 22:18:05.914693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.914665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kppsg" event={"ID":"f584b484-08cd-4488-8683-954e160667c4","Type":"ContainerStarted","Data":"51dca2ca4f259078f84f4a88632baca140f6135213161d0ee140f57544fb5435"}
Jul 16 22:18:05.930802 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.930590    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67968859f-8gm5q" podStartSLOduration=6.930574154 podStartE2EDuration="6.930574154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:05.930435389 +0000 UTC m=+42.250991461" watchObservedRunningTime="2026-07-16 22:18:05.930574154 +0000 UTC m=+42.251130226"
Jul 16 22:18:05.951669 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:05.951600    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kppsg" podStartSLOduration=8.55216246 podStartE2EDuration="40.951580523s" totalImagesPullingTime="32.399418063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:05.949993167 +0000 UTC m=+42.270549244" watchObservedRunningTime="2026-07-16 22:18:05.951580523 +0000 UTC m=+42.272136594"
Jul 16 22:18:06.596053 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.596015    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:06.596514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.596496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:06.596797 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.596774    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:06.601504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.600418    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 22:18:06.601504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.600429    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjst2\""
Jul 16 22:18:06.601504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.600719    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqtwh\""
Jul 16 22:18:06.601504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.600925    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 22:18:06.601504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.601112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 22:18:06.601504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:06.601314    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 22:18:12.652274 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.652244    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"]
Jul 16 22:18:12.678376 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.678349    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"]
Jul 16 22:18:12.678530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.678438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:12.684387 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.681307    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 16 22:18:12.684387 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.681402    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-twvwh\""
Jul 16 22:18:12.846089 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.846053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-67jrk\" (UID: \"5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:12.935384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.935305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5l475" event={"ID":"0a2a6174-b7bb-4201-82d5-8b4b59e930e1","Type":"ContainerStarted","Data":"317ef880c9e9e3f80dc347cc379d7ebcf2aa2d792d6977d2600a15582b1c538f"}
Jul 16 22:18:12.936716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.936692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6g7l" event={"ID":"7b8ce522-ef9d-4655-89a6-6e96b2fb0f12","Type":"ContainerStarted","Data":"b8d045d40667750d40f50f6a2354653bcdcc2acf58d09bcf669d66b06560fcbe"}
Jul 16 22:18:12.946648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.946619    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-67jrk\" (UID: \"5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:12.946806 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:12.946719    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 16 22:18:12.946806 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:12.946775    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc-tls-certificates podName:5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc nodeName:}" failed. No retries permitted until 2026-07-16 22:18:13.446761567 +0000 UTC m=+49.767317617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-67jrk" (UID: "5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 16 22:18:12.952279 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:12.952189    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q6g7l" podStartSLOduration=0.870524599 podStartE2EDuration="7.95217153s" totalImagesPullingTime="7.081646931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.951597244 +0000 UTC m=+49.272153317" watchObservedRunningTime="2026-07-16 22:18:12.95217153 +0000 UTC m=+49.272727583"
Jul 16 22:18:13.451288 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.451249    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-67jrk\" (UID: \"5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:13.453678 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.453648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-67jrk\" (UID: \"5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:13.591631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.591591    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:13.754408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.754299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"]
Jul 16 22:18:13.760740 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:13.759444    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee8dcb5_0a82_4b5e_ab0d_5b4abc415bcc.slice/crio-87851be7aedc526974bc00aebf9c021cb5451811d2d190151b8a196c2ad92f35 WatchSource:0}: Error finding container 87851be7aedc526974bc00aebf9c021cb5451811d2d190151b8a196c2ad92f35: Status 404 returned error can't find the container with id 87851be7aedc526974bc00aebf9c021cb5451811d2d190151b8a196c2ad92f35
Jul 16 22:18:13.942212 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.942177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk" event={"ID":"5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc","Type":"ContainerStarted","Data":"87851be7aedc526974bc00aebf9c021cb5451811d2d190151b8a196c2ad92f35"}
Jul 16 22:18:13.944497 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.944464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5l475" event={"ID":"0a2a6174-b7bb-4201-82d5-8b4b59e930e1","Type":"ContainerStarted","Data":"9c336cf31c97d700c2808675c5d2f66f48a3f36154a00f96179abde3a43f88f8"}
Jul 16 22:18:13.944745 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.944729    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:13.946936 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.946893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmz6v" event={"ID":"8e76aeb7-7a77-4efa-8798-780abcdf5801","Type":"ContainerStarted","Data":"d5064830c6e358338dcfd9b26d48bd513710bb582667b0f589094b1c2f7d2773"}
Jul 16 22:18:13.965108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:13.965050    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5l475" podStartSLOduration=2.827896627 podStartE2EDuration="9.965030141s" totalImagesPullingTime="7.137133514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:13.964242191 +0000 UTC m=+50.284798261" watchObservedRunningTime="2026-07-16 22:18:13.965030141 +0000 UTC m=+50.285586213"
Jul 16 22:18:17.961846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:17.961814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk" event={"ID":"5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc","Type":"ContainerStarted","Data":"471c8711b2ce3a97f393634b169fbe7f4bf193335db6d7b50d2c5b9e9fb66cd2"}
Jul 16 22:18:17.962227 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:17.962032    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:17.966641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:17.966622    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk"
Jul 16 22:18:17.978477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:17.978423    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-67jrk" podStartSLOduration=2.197326612 podStartE2EDuration="5.97840144s" totalImagesPullingTime="3.781074828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:17.978027351 +0000 UTC m=+54.298583424" watchObservedRunningTime="2026-07-16 22:18:17.97840144 +0000 UTC m=+54.298957513"
Jul 16 22:18:18.967520 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:18.967487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmz6v" event={"ID":"8e76aeb7-7a77-4efa-8798-780abcdf5801","Type":"ContainerStarted","Data":"03c41e225dd354ae38be29d980dc793c90795ed0716e107b677d4f983c626cd7"}
Jul 16 22:18:21.979083 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:21.979041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmz6v" event={"ID":"8e76aeb7-7a77-4efa-8798-780abcdf5801","Type":"ContainerStarted","Data":"389460ca11e8d4efdb27269f217c395d177ac6026837b2794f1010f9d3c447b3"}
Jul 16 22:18:21.997792 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:21.997741    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tmz6v" podStartSLOduration=1.393812257 podStartE2EDuration="16.99772629s" totalImagesPullingTime="15.603914033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:18:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:21.997461841 +0000 UTC m=+58.318017943" watchObservedRunningTime="2026-07-16 22:18:21.99772629 +0000 UTC m=+58.318282362"
Jul 16 22:18:23.952474 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:23.952443    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5l475"
Jul 16 22:18:25.884882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:25.884841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qxqw"
Jul 16 22:18:26.074111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.074080    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"]
Jul 16 22:18:26.076934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.076918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.079422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.079354    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 16 22:18:26.079714 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.079691    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 22:18:26.079807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.079787    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 16 22:18:26.079867 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.079814    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hncxw\""
Jul 16 22:18:26.079867 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.079847    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 22:18:26.079867 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.079855    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 22:18:26.099194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.099168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"]
Jul 16 22:18:26.099866 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.099843    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5hwsg"]
Jul 16 22:18:26.102543 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.102523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.104996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.104977    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 22:18:26.105108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.105088    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n649k\""
Jul 16 22:18:26.105430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.105415    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 22:18:26.106697 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.106673    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 22:18:26.149906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.149818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.149906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.149848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4d4q\" (UniqueName: \"kubernetes.io/projected/db47d412-8c36-4707-9adb-7af234f01ccf-kube-api-access-d4d4q\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.149906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.149884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-sys\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.149953    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-textfile\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-accelerators-collector-config\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a395eff-14dd-4748-908d-e996a653c9e8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv5jh\" (UniqueName: \"kubernetes.io/projected/0a395eff-14dd-4748-908d-e996a653c9e8-kube-api-access-tv5jh\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150080    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-wtmp\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a395eff-14dd-4748-908d-e996a653c9e8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.150157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db47d412-8c36-4707-9adb-7af234f01ccf-metrics-client-ca\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-tls\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-root\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.150367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.150254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a395eff-14dd-4748-908d-e996a653c9e8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.251360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db47d412-8c36-4707-9adb-7af234f01ccf-metrics-client-ca\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-tls\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-root\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a395eff-14dd-4748-908d-e996a653c9e8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4d4q\" (UniqueName: \"kubernetes.io/projected/db47d412-8c36-4707-9adb-7af234f01ccf-kube-api-access-d4d4q\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:26.251479    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-sys\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:26.251558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-tls podName:db47d412-8c36-4707-9adb-7af234f01ccf nodeName:}" failed. No retries permitted until 2026-07-16 22:18:26.751535828 +0000 UTC m=+63.072091880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-tls") pod "node-exporter-5hwsg" (UID: "db47d412-8c36-4707-9adb-7af234f01ccf") : secret "node-exporter-tls" not found
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-root\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.251622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-textfile\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-accelerators-collector-config\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-sys\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a395eff-14dd-4748-908d-e996a653c9e8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv5jh\" (UniqueName: \"kubernetes.io/projected/0a395eff-14dd-4748-908d-e996a653c9e8-kube-api-access-tv5jh\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-wtmp\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a395eff-14dd-4748-908d-e996a653c9e8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.251909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-textfile\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.252044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-wtmp\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.252074    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db47d412-8c36-4707-9adb-7af234f01ccf-metrics-client-ca\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.252278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-accelerators-collector-config\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.252608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.252590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a395eff-14dd-4748-908d-e996a653c9e8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.254062 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.254040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.254062 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.254053    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a395eff-14dd-4748-908d-e996a653c9e8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.254214 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.254095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a395eff-14dd-4748-908d-e996a653c9e8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.262152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.262113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv5jh\" (UniqueName: \"kubernetes.io/projected/0a395eff-14dd-4748-908d-e996a653c9e8-kube-api-access-tv5jh\") pod \"openshift-state-metrics-79f6958cdf-kngm7\" (UID: \"0a395eff-14dd-4748-908d-e996a653c9e8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.262922 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.262903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4d4q\" (UniqueName: \"kubernetes.io/projected/db47d412-8c36-4707-9adb-7af234f01ccf-kube-api-access-d4d4q\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.386261 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.386227    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"
Jul 16 22:18:26.512463 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.512428    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7"]
Jul 16 22:18:26.515226 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:26.515202    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a395eff_14dd_4748_908d_e996a653c9e8.slice/crio-4d51305722fc80621273745e754c3369400f9c4f8fb64c320f5f435e8b3b2657 WatchSource:0}: Error finding container 4d51305722fc80621273745e754c3369400f9c4f8fb64c320f5f435e8b3b2657: Status 404 returned error can't find the container with id 4d51305722fc80621273745e754c3369400f9c4f8fb64c320f5f435e8b3b2657
Jul 16 22:18:26.755395 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.755362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-tls\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.757620 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.757590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/db47d412-8c36-4707-9adb-7af234f01ccf-node-exporter-tls\") pod \"node-exporter-5hwsg\" (UID: \"db47d412-8c36-4707-9adb-7af234f01ccf\") " pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:26.923017 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.922989    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67968859f-8gm5q"
Jul 16 22:18:26.995701 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.995674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7" event={"ID":"0a395eff-14dd-4748-908d-e996a653c9e8","Type":"ContainerStarted","Data":"4051428684a92f45f73536ad46a824cd22a4379d66e452c40afbe6c678556e29"}
Jul 16 22:18:26.995850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.995739    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7" event={"ID":"0a395eff-14dd-4748-908d-e996a653c9e8","Type":"ContainerStarted","Data":"006f9c1bdf386055008b584dbb805822e60a3a1c1680ff97a9506885760dde71"}
Jul 16 22:18:26.995850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:26.995763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7" event={"ID":"0a395eff-14dd-4748-908d-e996a653c9e8","Type":"ContainerStarted","Data":"4d51305722fc80621273745e754c3369400f9c4f8fb64c320f5f435e8b3b2657"}
Jul 16 22:18:27.011021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.010958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5hwsg"
Jul 16 22:18:27.028858 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:27.028833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb47d412_8c36_4707_9adb_7af234f01ccf.slice/crio-f9900f0b48485c5de79f7acb96e1713a3653f189331ccb26925be937aa08ea30 WatchSource:0}: Error finding container f9900f0b48485c5de79f7acb96e1713a3653f189331ccb26925be937aa08ea30: Status 404 returned error can't find the container with id f9900f0b48485c5de79f7acb96e1713a3653f189331ccb26925be937aa08ea30
Jul 16 22:18:27.199342 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.199306    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:18:27.230254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.230218    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:18:27.230425 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.230371    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.232728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.232684    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 22:18:27.232728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.232702    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 22:18:27.232973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.232791    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 22:18:27.232973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.232830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2kc6t\""
Jul 16 22:18:27.233093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.232993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 22:18:27.233093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.233062    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 22:18:27.233093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.233073    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 22:18:27.233436 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.233418    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 22:18:27.233529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.233490    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 22:18:27.237909 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.237891    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 22:18:27.259989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.259958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8bj\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-kube-api-access-6p8bj\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260073    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260266 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260266 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260266 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-web-config\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-out\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.260543 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.260449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.361810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.361714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.361810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.361762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8bj\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-kube-api-access-6p8bj\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.361810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.361808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.361832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.361849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.361866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362279 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362336 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362336 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-web-config\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362439 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362439 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-out\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362439 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362580 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.362780 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.362756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.363095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.363069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.363252 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.363230    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.364976 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.364946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.365077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.364959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.365077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.365008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.365511 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.365488    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.365634 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.365608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.365740 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.365723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-out\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.365882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.365860    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-web-config\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.366164 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.366126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.366891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.366877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.370333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.370312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8bj\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-kube-api-access-6p8bj\") pod \"alertmanager-main-0\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.539636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.539597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:27.719021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:27.718860    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:18:27.721002 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:27.720975    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a2fd04_9c7f_4a4f_a58a_e03af3c1da18.slice/crio-6ca345d818a588c56731876c103e390ec376b50841809ffc615c2af7753d23c3 WatchSource:0}: Error finding container 6ca345d818a588c56731876c103e390ec376b50841809ffc615c2af7753d23c3: Status 404 returned error can't find the container with id 6ca345d818a588c56731876c103e390ec376b50841809ffc615c2af7753d23c3
Jul 16 22:18:28.001885 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:28.001842    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5hwsg" event={"ID":"db47d412-8c36-4707-9adb-7af234f01ccf","Type":"ContainerStarted","Data":"f9900f0b48485c5de79f7acb96e1713a3653f189331ccb26925be937aa08ea30"}
Jul 16 22:18:28.003261 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:28.003230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"6ca345d818a588c56731876c103e390ec376b50841809ffc615c2af7753d23c3"}
Jul 16 22:18:29.009777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:29.009738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7" event={"ID":"0a395eff-14dd-4748-908d-e996a653c9e8","Type":"ContainerStarted","Data":"73a8b23d27214738d515ba8d38874c0150eb3febec622ef0404bf0e80abca3d7"}
Jul 16 22:18:29.011501 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:29.011473    2564 generic.go:356] "Generic (PLEG): container finished" podID="db47d412-8c36-4707-9adb-7af234f01ccf" containerID="6f51945d3383f26b5087221f8b6f11860e0f1c122bb852b68830573298e17257" exitCode=0
Jul 16 22:18:29.011631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:29.011519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5hwsg" event={"ID":"db47d412-8c36-4707-9adb-7af234f01ccf","Type":"ContainerDied","Data":"6f51945d3383f26b5087221f8b6f11860e0f1c122bb852b68830573298e17257"}
Jul 16 22:18:29.028722 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:29.028668    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kngm7" podStartSLOduration=1.15447146 podStartE2EDuration="3.028647713s" totalImagesPullingTime="1.874176253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:29.028117564 +0000 UTC m=+65.348673637" watchObservedRunningTime="2026-07-16 22:18:29.028647713 +0000 UTC m=+65.349203785"
Jul 16 22:18:30.016699 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.016658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5hwsg" event={"ID":"db47d412-8c36-4707-9adb-7af234f01ccf","Type":"ContainerStarted","Data":"bfe1b3c7c84056a28681b4a76a15d3b81ceadb153d76b8648b60eeb18ee0fef3"}
Jul 16 22:18:30.017126 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.016722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5hwsg" event={"ID":"db47d412-8c36-4707-9adb-7af234f01ccf","Type":"ContainerStarted","Data":"9119facbf5c58587d14fabe4a18087743c4e6aa4033452936695ae7ff0ea4ddb"}
Jul 16 22:18:30.018078 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.018053    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e" exitCode=0
Jul 16 22:18:30.018189 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.018153    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"}
Jul 16 22:18:30.038209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.038163    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5hwsg" podStartSLOduration=2.482465679 podStartE2EDuration="4.038149881s" totalImagesPullingTime="1.489072301s" totalInitContainerRuntime="66.611901ms" isStatefulPod=true podCreationTimestamp="2026-07-16 22:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:30.036177046 +0000 UTC m=+66.356733109" watchObservedRunningTime="2026-07-16 22:18:30.038149881 +0000 UTC m=+66.358705949"
Jul 16 22:18:30.088743 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.088701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:30.089366 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.088936    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:30.089366 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.089030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:30.091426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.091407    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 22:18:30.091517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.091410    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 22:18:30.091517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.091410    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 22:18:30.101577 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.101555    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ec7c34d-4960-4611-8f14-9f5dac925696-original-pull-secret\") pod \"global-pull-secret-syncer-vxgdd\" (UID: \"2ec7c34d-4960-4611-8f14-9f5dac925696\") " pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:30.101651 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.101598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c5002e-be2b-48ff-9d4d-8a36d913f6b0-metrics-certs\") pod \"network-metrics-daemon-pkgx8\" (UID: \"68c5002e-be2b-48ff-9d4d-8a36d913f6b0\") " pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:30.101651 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.101608    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 22:18:30.112477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.112457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zknpf\" (UniqueName: \"kubernetes.io/projected/1b36dc0a-b970-442c-82cd-e35125a1bcb0-kube-api-access-zknpf\") pod \"network-check-target-sswr2\" (UID: \"1b36dc0a-b970-442c-82cd-e35125a1bcb0\") " pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:30.315282 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.315191    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqtwh\""
Jul 16 22:18:30.323146 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.323109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:30.325344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.325316    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjst2\""
Jul 16 22:18:30.331455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.331428    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vxgdd"
Jul 16 22:18:30.333561 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.333538    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkgx8"
Jul 16 22:18:30.464403 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.464370    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sswr2"]
Jul 16 22:18:30.467527 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:30.467483    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b36dc0a_b970_442c_82cd_e35125a1bcb0.slice/crio-43459d4f8842a37967055eb065c9f8b06ab8534f2673aa71cc8a1478b57da63d WatchSource:0}: Error finding container 43459d4f8842a37967055eb065c9f8b06ab8534f2673aa71cc8a1478b57da63d: Status 404 returned error can't find the container with id 43459d4f8842a37967055eb065c9f8b06ab8534f2673aa71cc8a1478b57da63d
Jul 16 22:18:30.692863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.692789    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vxgdd"]
Jul 16 22:18:30.696497 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:30.696462    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec7c34d_4960_4611_8f14_9f5dac925696.slice/crio-2bb4da1c1a03a2fba19ed807a15a9145b61d7139bf6e2cc1dbbe11f41a495691 WatchSource:0}: Error finding container 2bb4da1c1a03a2fba19ed807a15a9145b61d7139bf6e2cc1dbbe11f41a495691: Status 404 returned error can't find the container with id 2bb4da1c1a03a2fba19ed807a15a9145b61d7139bf6e2cc1dbbe11f41a495691
Jul 16 22:18:30.697091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.697064    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkgx8"]
Jul 16 22:18:30.700057 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:30.700039    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c5002e_be2b_48ff_9d4d_8a36d913f6b0.slice/crio-d3195cd724cd7638f3641a5af8f45194f046c4df99abf71a7e62d6a2341efcee WatchSource:0}: Error finding container d3195cd724cd7638f3641a5af8f45194f046c4df99abf71a7e62d6a2341efcee: Status 404 returned error can't find the container with id d3195cd724cd7638f3641a5af8f45194f046c4df99abf71a7e62d6a2341efcee
Jul 16 22:18:30.755187 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.755153    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f5898d9d4-9vrn4"]
Jul 16 22:18:30.758383 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.758368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.760922 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.760897    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 22:18:30.761676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.761661    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 22:18:30.762106 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.762087    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 22:18:30.762181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.762112    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 22:18:30.762181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.762086    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 22:18:30.762288 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.762260    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 22:18:30.762288 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.762264    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 22:18:30.762540 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.762527    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-frs6s\""
Jul 16 22:18:30.765986 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.765964    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 22:18:30.769153 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.769120    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5898d9d4-9vrn4"]
Jul 16 22:18:30.795932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.795901    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-console-config\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.795932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.795936    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-serving-cert\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.796155 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.795966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-trusted-ca-bundle\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.796155 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.796021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-service-ca\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.796155 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.796056    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-oauth-serving-cert\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.796155 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.796079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmdw7\" (UniqueName: \"kubernetes.io/projected/d52830ea-1605-4817-af98-00c84ce50997-kube-api-access-fmdw7\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.796155 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.796104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-oauth-config\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.857017 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.856988    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"]
Jul 16 22:18:30.859108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.859092    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:30.861886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.861867    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cnqz5\""
Jul 16 22:18:30.862027 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.861870    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 16 22:18:30.870253 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.870227    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"]
Jul 16 22:18:30.897241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-console-config\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-serving-cert\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-trusted-ca-bundle\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sjgb\" (UniqueName: \"kubernetes.io/projected/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-kube-api-access-5sjgb\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:30.897455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-service-ca\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-oauth-serving-cert\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmdw7\" (UniqueName: \"kubernetes.io/projected/d52830ea-1605-4817-af98-00c84ce50997-kube-api-access-fmdw7\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-oauth-config\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.897735 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.897471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:30.898037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.898017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-console-config\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.898159 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.898103    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-oauth-serving-cert\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.898229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.898199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-service-ca\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.898359 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.898335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-trusted-ca-bundle\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.899750 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.899732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-serving-cert\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.899835 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.899798    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-oauth-config\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.907516 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.907496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmdw7\" (UniqueName: \"kubernetes.io/projected/d52830ea-1605-4817-af98-00c84ce50997-kube-api-access-fmdw7\") pod \"console-7f5898d9d4-9vrn4\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") " pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:30.998816 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.998781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sjgb\" (UniqueName: \"kubernetes.io/projected/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-kube-api-access-5sjgb\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:30.998988 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:30.998839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:30.998988 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:30.998926    2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 16 22:18:30.998988 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:18:30.998980    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-monitoring-plugin-cert podName:72744ec4-fd34-49e5-bf0d-666fbb85ae1c nodeName:}" failed. No retries permitted until 2026-07-16 22:18:31.498964024 +0000 UTC m=+67.819520074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-g7lbc" (UID: "72744ec4-fd34-49e5-bf0d-666fbb85ae1c") : secret "monitoring-plugin-cert" not found
Jul 16 22:18:31.010378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.010354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sjgb\" (UniqueName: \"kubernetes.io/projected/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-kube-api-access-5sjgb\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:31.022786 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.022747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vxgdd" event={"ID":"2ec7c34d-4960-4611-8f14-9f5dac925696","Type":"ContainerStarted","Data":"2bb4da1c1a03a2fba19ed807a15a9145b61d7139bf6e2cc1dbbe11f41a495691"}
Jul 16 22:18:31.023859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.023834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkgx8" event={"ID":"68c5002e-be2b-48ff-9d4d-8a36d913f6b0","Type":"ContainerStarted","Data":"d3195cd724cd7638f3641a5af8f45194f046c4df99abf71a7e62d6a2341efcee"}
Jul 16 22:18:31.024921 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.024901    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sswr2" event={"ID":"1b36dc0a-b970-442c-82cd-e35125a1bcb0","Type":"ContainerStarted","Data":"43459d4f8842a37967055eb065c9f8b06ab8534f2673aa71cc8a1478b57da63d"}
Jul 16 22:18:31.068431 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.068388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:31.194905 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.194868    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5898d9d4-9vrn4"]
Jul 16 22:18:31.198456 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:31.198426    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52830ea_1605_4817_af98_00c84ce50997.slice/crio-bb14e5472522c09a64eb2b13a9cbcc80e35f5e256582b3119c8e826c6e38b96b WatchSource:0}: Error finding container bb14e5472522c09a64eb2b13a9cbcc80e35f5e256582b3119c8e826c6e38b96b: Status 404 returned error can't find the container with id bb14e5472522c09a64eb2b13a9cbcc80e35f5e256582b3119c8e826c6e38b96b
Jul 16 22:18:31.334998 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.334909    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"]
Jul 16 22:18:31.337807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.337777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.340908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.340858    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 16 22:18:31.341486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.341057    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 16 22:18:31.341486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.341317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 16 22:18:31.341486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.341372    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9hg8q\""
Jul 16 22:18:31.341486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.341459    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 16 22:18:31.341774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.341556    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 16 22:18:31.346043 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.345858    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 16 22:18:31.355014 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.354937    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"]
Jul 16 22:18:31.401837 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.401798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.401896    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-metrics-client-ca\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.401948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-secret-telemeter-client\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.401976    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-federate-client-tls\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.402002    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxqg\" (UniqueName: \"kubernetes.io/projected/3ced4254-2433-4a94-8c10-78702f1c1c6c-kube-api-access-mcxqg\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.402042    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.402085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-serving-certs-ca-bundle\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.402215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.402119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-telemeter-client-tls\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.503370 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.503271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-secret-telemeter-client\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.503370 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.503319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-federate-client-tls\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.503730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.503654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxqg\" (UniqueName: \"kubernetes.io/projected/3ced4254-2433-4a94-8c10-78702f1c1c6c-kube-api-access-mcxqg\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.503913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.503883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.504001 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.503936    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-serving-certs-ca-bundle\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.504001 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.503975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-telemeter-client-tls\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.504119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.504044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.504119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.504097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:31.505457 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.504154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-metrics-client-ca\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.505457 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.504897    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.505569 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.505473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-serving-certs-ca-bundle\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.505633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.505563    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ced4254-2433-4a94-8c10-78702f1c1c6c-metrics-client-ca\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.506807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.506706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-secret-telemeter-client\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.509065 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.509041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-telemeter-client-tls\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.510923 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.510888    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.511028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.510976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72744ec4-fd34-49e5-bf0d-666fbb85ae1c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-g7lbc\" (UID: \"72744ec4-fd34-49e5-bf0d-666fbb85ae1c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:31.511399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.511343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ced4254-2433-4a94-8c10-78702f1c1c6c-federate-client-tls\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.513695 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.513655    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxqg\" (UniqueName: \"kubernetes.io/projected/3ced4254-2433-4a94-8c10-78702f1c1c6c-kube-api-access-mcxqg\") pod \"telemeter-client-7d4d68bd5b-82hds\" (UID: \"3ced4254-2433-4a94-8c10-78702f1c1c6c\") " pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.650730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.650408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"
Jul 16 22:18:31.773005 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.772777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:31.832785 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:31.832683    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d4d68bd5b-82hds"]
Jul 16 22:18:32.033327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:32.033251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5898d9d4-9vrn4" event={"ID":"d52830ea-1605-4817-af98-00c84ce50997","Type":"ContainerStarted","Data":"bb14e5472522c09a64eb2b13a9cbcc80e35f5e256582b3119c8e826c6e38b96b"}
Jul 16 22:18:32.137358 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:32.137311    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ced4254_2433_4a94_8c10_78702f1c1c6c.slice/crio-b715683d8e9c9f0f83fa6d1253d6fd02579ebd4aa3a16b63415e18c205ef9312 WatchSource:0}: Error finding container b715683d8e9c9f0f83fa6d1253d6fd02579ebd4aa3a16b63415e18c205ef9312: Status 404 returned error can't find the container with id b715683d8e9c9f0f83fa6d1253d6fd02579ebd4aa3a16b63415e18c205ef9312
Jul 16 22:18:32.857827 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:32.857801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"]
Jul 16 22:18:32.870378 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:32.870335    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72744ec4_fd34_49e5_bf0d_666fbb85ae1c.slice/crio-27d70f007cdeec3a9cec6f8a0ec8243a619aea5cff062ef1b9fedb0e0a814730 WatchSource:0}: Error finding container 27d70f007cdeec3a9cec6f8a0ec8243a619aea5cff062ef1b9fedb0e0a814730: Status 404 returned error can't find the container with id 27d70f007cdeec3a9cec6f8a0ec8243a619aea5cff062ef1b9fedb0e0a814730
Jul 16 22:18:33.049079 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.049042    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"}
Jul 16 22:18:33.049452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.049198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"}
Jul 16 22:18:33.049452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.049243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"}
Jul 16 22:18:33.054412 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.054300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc" event={"ID":"72744ec4-fd34-49e5-bf0d-666fbb85ae1c","Type":"ContainerStarted","Data":"27d70f007cdeec3a9cec6f8a0ec8243a619aea5cff062ef1b9fedb0e0a814730"}
Jul 16 22:18:33.061299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.061266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds" event={"ID":"3ced4254-2433-4a94-8c10-78702f1c1c6c","Type":"ContainerStarted","Data":"b715683d8e9c9f0f83fa6d1253d6fd02579ebd4aa3a16b63415e18c205ef9312"}
Jul 16 22:18:33.065932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.064943    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkgx8" event={"ID":"68c5002e-be2b-48ff-9d4d-8a36d913f6b0","Type":"ContainerStarted","Data":"49d9d8455d35782f49218250e0e50741a608dc9b14e0af258e1354f1bcb2f51e"}
Jul 16 22:18:33.065932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.065000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkgx8" event={"ID":"68c5002e-be2b-48ff-9d4d-8a36d913f6b0","Type":"ContainerStarted","Data":"0a96d6f29432ca425581741750f0c02a249abf2a29415b429b5fe07d0904f6fd"}
Jul 16 22:18:33.083080 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:33.083017    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pkgx8" podStartSLOduration=66.091224322 podStartE2EDuration="1m8.082994147s" totalImagesPullingTime="1.991769825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:33.08142874 +0000 UTC m=+69.401984814" watchObservedRunningTime="2026-07-16 22:18:33.082994147 +0000 UTC m=+69.403550212"
Jul 16 22:18:34.079444 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.079279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"}
Jul 16 22:18:34.079444 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.079389    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"}
Jul 16 22:18:34.215319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.215277    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59f54f5fb8-np6gv"]
Jul 16 22:18:34.221151 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.221111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.230240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.230210    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59f54f5fb8-np6gv"]
Jul 16 22:18:34.329078 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgvzc\" (UniqueName: \"kubernetes.io/projected/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-kube-api-access-fgvzc\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.329078 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-oauth-config\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.329333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-config\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.329333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-oauth-serving-cert\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.329440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-serving-cert\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.329440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-trusted-ca-bundle\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.329440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.329420    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-service-ca\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430635 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-oauth-serving-cert\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430635 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-serving-cert\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-trusted-ca-bundle\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-service-ca\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgvzc\" (UniqueName: \"kubernetes.io/projected/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-kube-api-access-fgvzc\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-oauth-config\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.430853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.430769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-config\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.431497 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.431468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-oauth-serving-cert\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.431605 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.431513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-config\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.431605 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.431549    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-trusted-ca-bundle\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.431821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.431779    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-service-ca\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.433671 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.433638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-oauth-config\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.434194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.434170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-serving-cert\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.438957 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.438933    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgvzc\" (UniqueName: \"kubernetes.io/projected/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-kube-api-access-fgvzc\") pod \"console-59f54f5fb8-np6gv\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") " pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:34.532309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:34.532272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:38.625507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:38.624453    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59f54f5fb8-np6gv"]
Jul 16 22:18:38.631081 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:38.631038    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0b5cc1_21da_435d_9d1a_baabe8e189d4.slice/crio-b6efba97999755b65488fba966769a6542adac86bdd64fef116e3441cb89381d WatchSource:0}: Error finding container b6efba97999755b65488fba966769a6542adac86bdd64fef116e3441cb89381d: Status 404 returned error can't find the container with id b6efba97999755b65488fba966769a6542adac86bdd64fef116e3441cb89381d
Jul 16 22:18:39.108433 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.108343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5898d9d4-9vrn4" event={"ID":"d52830ea-1605-4817-af98-00c84ce50997","Type":"ContainerStarted","Data":"6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3"}
Jul 16 22:18:39.109814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.109786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sswr2" event={"ID":"1b36dc0a-b970-442c-82cd-e35125a1bcb0","Type":"ContainerStarted","Data":"e602eaf13007134157e9313d2c95d7ad3b3ecb5f55b3ca0049418c3a001ec3a0"}
Jul 16 22:18:39.109937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.109854    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:18:39.111241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.111219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f54f5fb8-np6gv" event={"ID":"fa0b5cc1-21da-435d-9d1a-baabe8e189d4","Type":"ContainerStarted","Data":"4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6"}
Jul 16 22:18:39.111364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.111255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f54f5fb8-np6gv" event={"ID":"fa0b5cc1-21da-435d-9d1a-baabe8e189d4","Type":"ContainerStarted","Data":"b6efba97999755b65488fba966769a6542adac86bdd64fef116e3441cb89381d"}
Jul 16 22:18:39.115264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.115242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vxgdd" event={"ID":"2ec7c34d-4960-4611-8f14-9f5dac925696","Type":"ContainerStarted","Data":"34bafced31814080d58ccffca306d7be03e87cbd6158f842a6ccdb37e6c087d8"}
Jul 16 22:18:39.118540 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.118515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerStarted","Data":"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"}
Jul 16 22:18:39.120212 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.120193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc" event={"ID":"72744ec4-fd34-49e5-bf0d-666fbb85ae1c","Type":"ContainerStarted","Data":"fc46938e38b45f687d66b7af473122ec1789673041327997e9dd378290217b63"}
Jul 16 22:18:39.120308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.120294    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:39.122483 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.122459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds" event={"ID":"3ced4254-2433-4a94-8c10-78702f1c1c6c","Type":"ContainerStarted","Data":"4e51f7b06c75270e086d1578865a54afd8cdc31178fbaa65def780ac36d3fe7d"}
Jul 16 22:18:39.122568 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.122508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds" event={"ID":"3ced4254-2433-4a94-8c10-78702f1c1c6c","Type":"ContainerStarted","Data":"0d6f641e36f260ce5793b9bfb271b2f81f8d597743acf2d6f1795d783fe09965"}
Jul 16 22:18:39.122568 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.122543    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds" event={"ID":"3ced4254-2433-4a94-8c10-78702f1c1c6c","Type":"ContainerStarted","Data":"4f00992a8f37cd0c2873c458079b5c4168566baeb5cc50ab6e1a3afe356f10ce"}
Jul 16 22:18:39.125939 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.125920    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc"
Jul 16 22:18:39.126390 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.126352    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f5898d9d4-9vrn4" podStartSLOduration=1.858761946 podStartE2EDuration="9.126340879s" totalImagesPullingTime="7.267578933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.125996297 +0000 UTC m=+75.446552371" watchObservedRunningTime="2026-07-16 22:18:39.126340879 +0000 UTC m=+75.446896951"
Jul 16 22:18:39.149591 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.149539    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d4d68bd5b-82hds" podStartSLOduration=1.820743988 podStartE2EDuration="8.149522771s" totalImagesPullingTime="6.328778783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.147855335 +0000 UTC m=+75.468411407" watchObservedRunningTime="2026-07-16 22:18:39.149522771 +0000 UTC m=+75.470078878"
Jul 16 22:18:39.166760 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.166693    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59f54f5fb8-np6gv" podStartSLOduration=5.16667824 podStartE2EDuration="5.16667824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.165411305 +0000 UTC m=+75.485967374" watchObservedRunningTime="2026-07-16 22:18:39.16667824 +0000 UTC m=+75.487234289"
Jul 16 22:18:39.181859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.181804    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vxgdd" podStartSLOduration=66.411588011 podStartE2EDuration="1m14.181789834s" totalImagesPullingTime="7.770201823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.181729479 +0000 UTC m=+75.502285550" watchObservedRunningTime="2026-07-16 22:18:39.181789834 +0000 UTC m=+75.502345903"
Jul 16 22:18:39.197594 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.197542    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-g7lbc" podStartSLOduration=3.570955852 podStartE2EDuration="9.197527284s" totalImagesPullingTime="5.626571432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.196103316 +0000 UTC m=+75.516659452" watchObservedRunningTime="2026-07-16 22:18:39.197527284 +0000 UTC m=+75.518083355"
Jul 16 22:18:39.211663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.211618    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sswr2" podStartSLOduration=66.211671998 podStartE2EDuration="1m14.21160242s" totalImagesPullingTime="7.999930422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.211158339 +0000 UTC m=+75.531714415" watchObservedRunningTime="2026-07-16 22:18:39.21160242 +0000 UTC m=+75.532158492"
Jul 16 22:18:39.242185 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.242113    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.7278158230000003 podStartE2EDuration="12.242097118s" totalImagesPullingTime="9.492704653s" totalInitContainerRuntime="21.576642ms" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:39.240255372 +0000 UTC m=+75.560811447" watchObservedRunningTime="2026-07-16 22:18:39.242097118 +0000 UTC m=+75.562653192"
Jul 16 22:18:39.914457 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.914413    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5898d9d4-9vrn4"]
Jul 16 22:18:39.950449 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.950417    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75c8864975-nmprh"]
Jul 16 22:18:39.953200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.953185    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:39.967156 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:39.967117    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75c8864975-nmprh"]
Jul 16 22:18:40.086928 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.086888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-oauth-config\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.086928 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.086934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-trusted-ca-bundle\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.087188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.086955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-serving-cert\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.087188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.087030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-console-config\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.087188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.087077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-service-ca\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.087188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.087098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-oauth-serving-cert\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.087188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.087169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/698acefb-c493-4a94-9156-dcd531013569-kube-api-access-nv8q7\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.188122 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-service-ca\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.188122 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188070    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-oauth-serving-cert\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.188381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/698acefb-c493-4a94-9156-dcd531013569-kube-api-access-nv8q7\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.188381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-oauth-config\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.188486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-trusted-ca-bundle\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.188560 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-serving-cert\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.189716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188874    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-service-ca\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.189716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.188918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-oauth-serving-cert\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.189716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.189439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-console-config\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.190262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.190225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-trusted-ca-bundle\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.190563 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.190542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-console-config\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.191432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.191411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-oauth-config\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.192115 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.192093    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-serving-cert\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.200431 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.200406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/698acefb-c493-4a94-9156-dcd531013569-kube-api-access-nv8q7\") pod \"console-75c8864975-nmprh\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") " pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.262581 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.262545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:40.400103 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:40.400075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75c8864975-nmprh"]
Jul 16 22:18:40.402399 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:18:40.402375    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698acefb_c493_4a94_9156_dcd531013569.slice/crio-173b8b75f892ce4816bf7d5dca6f2c5d938ccd4dd05f0f5ae997793d4de72eed WatchSource:0}: Error finding container 173b8b75f892ce4816bf7d5dca6f2c5d938ccd4dd05f0f5ae997793d4de72eed: Status 404 returned error can't find the container with id 173b8b75f892ce4816bf7d5dca6f2c5d938ccd4dd05f0f5ae997793d4de72eed
Jul 16 22:18:41.068698 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.068662    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:41.069059 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.068753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:41.073690 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.073667    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:41.132401 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.132359    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c8864975-nmprh" event={"ID":"698acefb-c493-4a94-9156-dcd531013569","Type":"ContainerStarted","Data":"9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03"}
Jul 16 22:18:41.132572 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.132422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c8864975-nmprh" event={"ID":"698acefb-c493-4a94-9156-dcd531013569","Type":"ContainerStarted","Data":"173b8b75f892ce4816bf7d5dca6f2c5d938ccd4dd05f0f5ae997793d4de72eed"}
Jul 16 22:18:41.136584 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.136554    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:18:41.152150 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:41.152070    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75c8864975-nmprh" podStartSLOduration=2.152049046 podStartE2EDuration="2.152049046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:41.150738994 +0000 UTC m=+77.471295061" watchObservedRunningTime="2026-07-16 22:18:41.152049046 +0000 UTC m=+77.472605118"
Jul 16 22:18:44.533069 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:44.533028    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:44.533473 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:44.533153    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:44.537854 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:44.537829    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:45.152200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:45.152167    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:18:50.263340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:50.263208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:50.263340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:50.263275    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:50.267948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:50.267926    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:51.172211 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:51.172184    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:18:51.247677 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:18:51.247646    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59f54f5fb8-np6gv"]
Jul 16 22:19:06.153284 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:06.153202    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f5898d9d4-9vrn4" podUID="d52830ea-1605-4817-af98-00c84ce50997" containerName="console" containerID="cri-o://6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3" gracePeriod=15
Jul 16 22:19:06.934921 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:06.934893    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:19:07.015168 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015117    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-service-ca\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015180    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-oauth-config\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015217    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-serving-cert\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015236    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-console-config\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015261    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-trusted-ca-bundle\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015300    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmdw7\" (UniqueName: \"kubernetes.io/projected/d52830ea-1605-4817-af98-00c84ce50997-kube-api-access-fmdw7\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015322    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-oauth-serving-cert\") pod \"d52830ea-1605-4817-af98-00c84ce50997\" (UID: \"d52830ea-1605-4817-af98-00c84ce50997\") "
Jul 16 22:19:07.015638 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015547    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-service-ca" (OuterVolumeSpecName: "service-ca") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:07.015878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015717    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-console-config" (OuterVolumeSpecName: "console-config") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:07.015878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015775    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:07.015878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.015826    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:07.017633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.017601    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:07.017722 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.017629    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52830ea-1605-4817-af98-00c84ce50997-kube-api-access-fmdw7" (OuterVolumeSpecName: "kube-api-access-fmdw7") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "kube-api-access-fmdw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:19:07.017722 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.017649    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d52830ea-1605-4817-af98-00c84ce50997" (UID: "d52830ea-1605-4817-af98-00c84ce50997"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.115954    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.115983    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-console-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.115992    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.116002    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmdw7\" (UniqueName: \"kubernetes.io/projected/d52830ea-1605-4817-af98-00c84ce50997-kube-api-access-fmdw7\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.116011    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-oauth-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.116019    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d52830ea-1605-4817-af98-00c84ce50997-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.116040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.116027    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d52830ea-1605-4817-af98-00c84ce50997-console-oauth-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:07.230688 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.230656    2564 generic.go:356] "Generic (PLEG): container finished" podID="d52830ea-1605-4817-af98-00c84ce50997" containerID="6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3" exitCode=0
Jul 16 22:19:07.231091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.230699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5898d9d4-9vrn4" event={"ID":"d52830ea-1605-4817-af98-00c84ce50997","Type":"ContainerDied","Data":"6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3"}
Jul 16 22:19:07.231091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.230721    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5898d9d4-9vrn4"
Jul 16 22:19:07.231091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.230776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5898d9d4-9vrn4" event={"ID":"d52830ea-1605-4817-af98-00c84ce50997","Type":"ContainerDied","Data":"bb14e5472522c09a64eb2b13a9cbcc80e35f5e256582b3119c8e826c6e38b96b"}
Jul 16 22:19:07.231091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.230797    2564 scope.go:117] "RemoveContainer" containerID="6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3"
Jul 16 22:19:07.239494 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.239474    2564 scope.go:117] "RemoveContainer" containerID="6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3"
Jul 16 22:19:07.239735 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:07.239714    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3\": container with ID starting with 6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3 not found: ID does not exist" containerID="6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3"
Jul 16 22:19:07.239810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.239746    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3"} err="failed to get container status \"6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3\": rpc error: code = NotFound desc = could not find container \"6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3\": container with ID starting with 6ba1d1dbad053db8af5d43840fdb3c50290437d2040e0d386e3d82068adeb6a3 not found: ID does not exist"
Jul 16 22:19:07.251840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.251815    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5898d9d4-9vrn4"]
Jul 16 22:19:07.256464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.256442    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f5898d9d4-9vrn4"]
Jul 16 22:19:07.600674 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:07.600642    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d52830ea-1605-4817-af98-00c84ce50997" path="/var/lib/kubelet/pods/d52830ea-1605-4817-af98-00c84ce50997/volumes"
Jul 16 22:19:10.129274 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:10.129243    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sswr2"
Jul 16 22:19:16.267956 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.267910    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59f54f5fb8-np6gv" podUID="fa0b5cc1-21da-435d-9d1a-baabe8e189d4" containerName="console" containerID="cri-o://4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6" gracePeriod=15
Jul 16 22:19:16.513421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.513397    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:19:16.589571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589482    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-serving-cert\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.589571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589541    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-oauth-config\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.589809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589584    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgvzc\" (UniqueName: \"kubernetes.io/projected/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-kube-api-access-fgvzc\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.589809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589626    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-config\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.589809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589652    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-oauth-serving-cert\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.589809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589675    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-trusted-ca-bundle\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.589809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.589709    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-service-ca\") pod \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\" (UID: \"fa0b5cc1-21da-435d-9d1a-baabe8e189d4\") "
Jul 16 22:19:16.590195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.590160    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:16.590304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.590167    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-config" (OuterVolumeSpecName: "console-config") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:16.590304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.590194    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:16.590304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.590244    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-service-ca" (OuterVolumeSpecName: "service-ca") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:16.591846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.591822    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-kube-api-access-fgvzc" (OuterVolumeSpecName: "kube-api-access-fgvzc") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "kube-api-access-fgvzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:19:16.591953 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.591824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:16.591953 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.591892    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fa0b5cc1-21da-435d-9d1a-baabe8e189d4" (UID: "fa0b5cc1-21da-435d-9d1a-baabe8e189d4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:16.691066 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691016    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.691066 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691059    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-oauth-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.691066 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691073    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgvzc\" (UniqueName: \"kubernetes.io/projected/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-kube-api-access-fgvzc\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.691324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691085    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-console-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.691324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691098    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-oauth-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.691324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691109    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.691324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:16.691119    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa0b5cc1-21da-435d-9d1a-baabe8e189d4-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:17.270485 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.270448    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa0b5cc1-21da-435d-9d1a-baabe8e189d4" containerID="4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6" exitCode=0
Jul 16 22:19:17.270984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.270498    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f54f5fb8-np6gv"
Jul 16 22:19:17.270984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.270503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f54f5fb8-np6gv" event={"ID":"fa0b5cc1-21da-435d-9d1a-baabe8e189d4","Type":"ContainerDied","Data":"4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6"}
Jul 16 22:19:17.270984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.270624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f54f5fb8-np6gv" event={"ID":"fa0b5cc1-21da-435d-9d1a-baabe8e189d4","Type":"ContainerDied","Data":"b6efba97999755b65488fba966769a6542adac86bdd64fef116e3441cb89381d"}
Jul 16 22:19:17.270984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.270651    2564 scope.go:117] "RemoveContainer" containerID="4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6"
Jul 16 22:19:17.279370 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.279351    2564 scope.go:117] "RemoveContainer" containerID="4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6"
Jul 16 22:19:17.279627 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:17.279608    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6\": container with ID starting with 4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6 not found: ID does not exist" containerID="4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6"
Jul 16 22:19:17.279694 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.279636    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6"} err="failed to get container status \"4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6\": rpc error: code = NotFound desc = could not find container \"4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6\": container with ID starting with 4f1975151c62abf4fa7efe209158ef908adb5bb4eb2704b78d369f63f3832ee6 not found: ID does not exist"
Jul 16 22:19:17.293781 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.293753    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59f54f5fb8-np6gv"]
Jul 16 22:19:17.296564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.296543    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59f54f5fb8-np6gv"]
Jul 16 22:19:17.600926 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:17.600846    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa0b5cc1-21da-435d-9d1a-baabe8e189d4" path="/var/lib/kubelet/pods/fa0b5cc1-21da-435d-9d1a-baabe8e189d4/volumes"
Jul 16 22:19:46.487207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.487164    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:19:46.487770 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.487532    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="alertmanager" containerID="cri-o://1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf" gracePeriod=120
Jul 16 22:19:46.487770 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.487641    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy" containerID="cri-o://5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed" gracePeriod=120
Jul 16 22:19:46.488044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.487681    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-web" containerID="cri-o://754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622" gracePeriod=120
Jul 16 22:19:46.488313 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.487603    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-metric" containerID="cri-o://94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c" gracePeriod=120
Jul 16 22:19:46.488428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.488183    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="config-reloader" containerID="cri-o://99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3" gracePeriod=120
Jul 16 22:19:46.488428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:46.488385    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="prom-label-proxy" containerID="cri-o://5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2" gracePeriod=120
Jul 16 22:19:47.382161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382110    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2" exitCode=0
Jul 16 22:19:47.382161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382150    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed" exitCode=0
Jul 16 22:19:47.382161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382157    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3" exitCode=0
Jul 16 22:19:47.382161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382163    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf" exitCode=0
Jul 16 22:19:47.382466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"}
Jul 16 22:19:47.382466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"}
Jul 16 22:19:47.382466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382315    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"}
Jul 16 22:19:47.382466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.382351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"}
Jul 16 22:19:47.757926 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.757903    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:47.837949 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.837916    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-metrics-client-ca\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.837949 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.837953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-cluster-tls-config\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838007    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-web-config\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838048    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838076    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-main-tls\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838106    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-out\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838129    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-tls-assets\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838169    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p8bj\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-kube-api-access-6p8bj\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838213    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-main-db\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838243    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-web\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-trusted-ca-bundle\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-volume\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy\") pod \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\" (UID: \"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18\") "
Jul 16 22:19:47.838736 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.838595    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:47.839146 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.839096    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:19:47.839505 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.839479    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:47.841296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.841251    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.841399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.841356    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.841908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.841877    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-kube-api-access-6p8bj" (OuterVolumeSpecName: "kube-api-access-6p8bj") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "kube-api-access-6p8bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:19:47.842000 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.841955    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-out" (OuterVolumeSpecName: "config-out") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:19:47.842230 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.842178    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.842306 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.842258    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:19:47.842532 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.842508    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.842696 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.842681    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-volume" (OuterVolumeSpecName: "config-volume") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.846403 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.846378    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.852366 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.852345    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-web-config" (OuterVolumeSpecName: "web-config") pod "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" (UID: "a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939894    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939929    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-main-tls\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939941    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-out\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939951    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-tls-assets\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939960    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p8bj\" (UniqueName: \"kubernetes.io/projected/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-kube-api-access-6p8bj\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939968    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-main-db\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.939980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939979    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.940305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939990    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.940305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.939998    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-config-volume\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.940305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.940007    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.940305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.940015    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-metrics-client-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.940305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.940023    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-cluster-tls-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:47.940305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:47.940031    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18-web-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:19:48.389240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389207    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c" exitCode=0
Jul 16 22:19:48.389240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389233    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerID="754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622" exitCode=0
Jul 16 22:19:48.389498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"}
Jul 16 22:19:48.389498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389300    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.389498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"}
Jul 16 22:19:48.389498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18","Type":"ContainerDied","Data":"6ca345d818a588c56731876c103e390ec376b50841809ffc615c2af7753d23c3"}
Jul 16 22:19:48.389498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.389373    2564 scope.go:117] "RemoveContainer" containerID="5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"
Jul 16 22:19:48.397383 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.397353    2564 scope.go:117] "RemoveContainer" containerID="94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"
Jul 16 22:19:48.404214 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.404197    2564 scope.go:117] "RemoveContainer" containerID="5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"
Jul 16 22:19:48.410799 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.410777    2564 scope.go:117] "RemoveContainer" containerID="754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"
Jul 16 22:19:48.417301 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.417284    2564 scope.go:117] "RemoveContainer" containerID="99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"
Jul 16 22:19:48.419889 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.419868    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:19:48.424974 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.424951    2564 scope.go:117] "RemoveContainer" containerID="1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"
Jul 16 22:19:48.428623 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.428598    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:19:48.432547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.432527    2564 scope.go:117] "RemoveContainer" containerID="0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"
Jul 16 22:19:48.439408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.439392    2564 scope.go:117] "RemoveContainer" containerID="5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"
Jul 16 22:19:48.439671 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.439642    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2\": container with ID starting with 5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2 not found: ID does not exist" containerID="5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"
Jul 16 22:19:48.439731 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.439680    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"} err="failed to get container status \"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2\": rpc error: code = NotFound desc = could not find container \"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2\": container with ID starting with 5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2 not found: ID does not exist"
Jul 16 22:19:48.439731 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.439700    2564 scope.go:117] "RemoveContainer" containerID="94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"
Jul 16 22:19:48.439950 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.439933    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c\": container with ID starting with 94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c not found: ID does not exist" containerID="94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"
Jul 16 22:19:48.439989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.439959    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"} err="failed to get container status \"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c\": rpc error: code = NotFound desc = could not find container \"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c\": container with ID starting with 94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c not found: ID does not exist"
Jul 16 22:19:48.439989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.439976    2564 scope.go:117] "RemoveContainer" containerID="5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"
Jul 16 22:19:48.440216 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.440199    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed\": container with ID starting with 5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed not found: ID does not exist" containerID="5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"
Jul 16 22:19:48.440275 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440225    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"} err="failed to get container status \"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed\": rpc error: code = NotFound desc = could not find container \"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed\": container with ID starting with 5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed not found: ID does not exist"
Jul 16 22:19:48.440275 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440245    2564 scope.go:117] "RemoveContainer" containerID="754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"
Jul 16 22:19:48.440470 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.440456    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622\": container with ID starting with 754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622 not found: ID does not exist" containerID="754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"
Jul 16 22:19:48.440510 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440474    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"} err="failed to get container status \"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622\": rpc error: code = NotFound desc = could not find container \"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622\": container with ID starting with 754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622 not found: ID does not exist"
Jul 16 22:19:48.440510 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440486    2564 scope.go:117] "RemoveContainer" containerID="99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"
Jul 16 22:19:48.440675 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.440660    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3\": container with ID starting with 99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3 not found: ID does not exist" containerID="99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"
Jul 16 22:19:48.440715 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440680    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"} err="failed to get container status \"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3\": rpc error: code = NotFound desc = could not find container \"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3\": container with ID starting with 99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3 not found: ID does not exist"
Jul 16 22:19:48.440715 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440698    2564 scope.go:117] "RemoveContainer" containerID="1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"
Jul 16 22:19:48.440920 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.440904    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf\": container with ID starting with 1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf not found: ID does not exist" containerID="1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"
Jul 16 22:19:48.440970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440922    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"} err="failed to get container status \"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf\": rpc error: code = NotFound desc = could not find container \"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf\": container with ID starting with 1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf not found: ID does not exist"
Jul 16 22:19:48.440970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.440935    2564 scope.go:117] "RemoveContainer" containerID="0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"
Jul 16 22:19:48.441124 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:19:48.441105    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e\": container with ID starting with 0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e not found: ID does not exist" containerID="0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"
Jul 16 22:19:48.441181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441127    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"} err="failed to get container status \"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e\": rpc error: code = NotFound desc = could not find container \"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e\": container with ID starting with 0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e not found: ID does not exist"
Jul 16 22:19:48.441181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441155    2564 scope.go:117] "RemoveContainer" containerID="5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"
Jul 16 22:19:48.441364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441347    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2"} err="failed to get container status \"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2\": rpc error: code = NotFound desc = could not find container \"5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2\": container with ID starting with 5d0ac37f4a7d0d86b294c8704a5aa6712695cab48361513c9252cfbef2b102b2 not found: ID does not exist"
Jul 16 22:19:48.441364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441363    2564 scope.go:117] "RemoveContainer" containerID="94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"
Jul 16 22:19:48.441543 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441527    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c"} err="failed to get container status \"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c\": rpc error: code = NotFound desc = could not find container \"94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c\": container with ID starting with 94ca88fcf1d7ca4b7ac233fa15689df1e797c47975827ddcefbc4f44c333aa8c not found: ID does not exist"
Jul 16 22:19:48.441590 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441544    2564 scope.go:117] "RemoveContainer" containerID="5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"
Jul 16 22:19:48.441724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441709    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed"} err="failed to get container status \"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed\": rpc error: code = NotFound desc = could not find container \"5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed\": container with ID starting with 5003ba86ed57660187c0c603ae80d6551072a4201fa39e4df7aec4216c93bfed not found: ID does not exist"
Jul 16 22:19:48.441764 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441725    2564 scope.go:117] "RemoveContainer" containerID="754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"
Jul 16 22:19:48.441900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441885    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622"} err="failed to get container status \"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622\": rpc error: code = NotFound desc = could not find container \"754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622\": container with ID starting with 754cc929e4e52a47eafbe68d977291d459d45d4e501026f8878afd0a721c9622 not found: ID does not exist"
Jul 16 22:19:48.441941 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.441902    2564 scope.go:117] "RemoveContainer" containerID="99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"
Jul 16 22:19:48.442065 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.442049    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3"} err="failed to get container status \"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3\": rpc error: code = NotFound desc = could not find container \"99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3\": container with ID starting with 99c33bcfdfd94fec2208c989fd4eab9a12171f58b268ab1ac9cd44b4f44de5d3 not found: ID does not exist"
Jul 16 22:19:48.442065 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.442064    2564 scope.go:117] "RemoveContainer" containerID="1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"
Jul 16 22:19:48.442242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.442225    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf"} err="failed to get container status \"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf\": rpc error: code = NotFound desc = could not find container \"1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf\": container with ID starting with 1a15a8f6a649af7e030cc434a4bed9aa481129eafdad336f11eff93b8ff17fbf not found: ID does not exist"
Jul 16 22:19:48.442282 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.442242    2564 scope.go:117] "RemoveContainer" containerID="0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"
Jul 16 22:19:48.442416 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.442400    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e"} err="failed to get container status \"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e\": rpc error: code = NotFound desc = could not find container \"0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e\": container with ID starting with 0f6e449ecc49a34430bebebad2381dc282b83d3d29a358b9d728423af16ceb7e not found: ID does not exist"
Jul 16 22:19:48.459802 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.459778    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:19:48.460021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460008    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="init-config-reloader"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460024    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="init-config-reloader"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460034    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="alertmanager"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460039    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="alertmanager"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460050    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa0b5cc1-21da-435d-9d1a-baabe8e189d4" containerName="console"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460055    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa0b5cc1-21da-435d-9d1a-baabe8e189d4" containerName="console"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460063    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-web"
Jul 16 22:19:48.460068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460070    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-web"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460080    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52830ea-1605-4817-af98-00c84ce50997" containerName="console"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460087    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52830ea-1605-4817-af98-00c84ce50997" containerName="console"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460095    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="config-reloader"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460100    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="config-reloader"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="prom-label-proxy"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460112    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="prom-label-proxy"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460118    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460143    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460149    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-metric"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460155    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-metric"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460195    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="config-reloader"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460202    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460208    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="prom-label-proxy"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460215    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-metric"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460219    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d52830ea-1605-4817-af98-00c84ce50997" containerName="console"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460224    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa0b5cc1-21da-435d-9d1a-baabe8e189d4" containerName="console"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460229    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="alertmanager"
Jul 16 22:19:48.460298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.460234    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" containerName="kube-rbac-proxy-web"
Jul 16 22:19:48.463932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.463917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.469042 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.469024    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 22:19:48.469332 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.469316    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 22:19:48.469423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.469354    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2kc6t\""
Jul 16 22:19:48.469917 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.469834    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 22:19:48.470000 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.469915    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 22:19:48.470160 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.470147    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 22:19:48.470237 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.470149    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 22:19:48.470237 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.470178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 22:19:48.470408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.470392    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 22:19:48.476270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.476252    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 22:19:48.487633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.487609    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:19:48.544657 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f48b920-277d-4cb9-a2bf-53f7dea480c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.544657 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544670    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6f48b920-277d-4cb9-a2bf-53f7dea480c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.544883 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544698    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9dt5\" (UniqueName: \"kubernetes.io/projected/6f48b920-277d-4cb9-a2bf-53f7dea480c4-kube-api-access-l9dt5\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.544883 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.544883 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f48b920-277d-4cb9-a2bf-53f7dea480c4-config-out\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.544883 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544899    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f48b920-277d-4cb9-a2bf-53f7dea480c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-web-config\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.544975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.545022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.545068    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.545095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.545311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.545120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f48b920-277d-4cb9-a2bf-53f7dea480c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646417    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646436    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f48b920-277d-4cb9-a2bf-53f7dea480c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f48b920-277d-4cb9-a2bf-53f7dea480c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6f48b920-277d-4cb9-a2bf-53f7dea480c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9dt5\" (UniqueName: \"kubernetes.io/projected/6f48b920-277d-4cb9-a2bf-53f7dea480c4-kube-api-access-l9dt5\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646576    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f48b920-277d-4cb9-a2bf-53f7dea480c4-config-out\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f48b920-277d-4cb9-a2bf-53f7dea480c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-web-config\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.646863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.646733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.647264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.647091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6f48b920-277d-4cb9-a2bf-53f7dea480c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.647528 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.647493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f48b920-277d-4cb9-a2bf-53f7dea480c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.648167 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.648120    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f48b920-277d-4cb9-a2bf-53f7dea480c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.649814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.649591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.649814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.649676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.649814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.649745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f48b920-277d-4cb9-a2bf-53f7dea480c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.649814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.649764    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.649814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.649789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.650178 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.650069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.650436 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.650418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.650545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.650527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f48b920-277d-4cb9-a2bf-53f7dea480c4-config-out\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.651530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.651510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f48b920-277d-4cb9-a2bf-53f7dea480c4-web-config\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.661822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.661802    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9dt5\" (UniqueName: \"kubernetes.io/projected/6f48b920-277d-4cb9-a2bf-53f7dea480c4-kube-api-access-l9dt5\") pod \"alertmanager-main-0\" (UID: \"6f48b920-277d-4cb9-a2bf-53f7dea480c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.773335 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.773297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:19:48.909752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:48.909696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:19:48.911803 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:19:48.911780    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f48b920_277d_4cb9_a2bf_53f7dea480c4.slice/crio-dd568d89d6601b26cf6c98292e04b25d3626e5c35eb0da4cc54dbbdd335f83d0 WatchSource:0}: Error finding container dd568d89d6601b26cf6c98292e04b25d3626e5c35eb0da4cc54dbbdd335f83d0: Status 404 returned error can't find the container with id dd568d89d6601b26cf6c98292e04b25d3626e5c35eb0da4cc54dbbdd335f83d0
Jul 16 22:19:49.396011 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:49.395979    2564 generic.go:356] "Generic (PLEG): container finished" podID="6f48b920-277d-4cb9-a2bf-53f7dea480c4" containerID="2c09771c96b5c9b5aa70663a63c981bc5b6205fac3f4bef0dca974ebdeb1b002" exitCode=0
Jul 16 22:19:49.396166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:49.396039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerDied","Data":"2c09771c96b5c9b5aa70663a63c981bc5b6205fac3f4bef0dca974ebdeb1b002"}
Jul 16 22:19:49.396166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:49.396090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"dd568d89d6601b26cf6c98292e04b25d3626e5c35eb0da4cc54dbbdd335f83d0"}
Jul 16 22:19:49.600231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:49.600201    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18" path="/var/lib/kubelet/pods/a4a2fd04-9c7f-4a4f-a58a-e03af3c1da18/volumes"
Jul 16 22:19:50.402933 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.402897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"76188cba9f3198b71371cd841cc6063f450140b7b4ddfb658dc8ed8cc587ba99"}
Jul 16 22:19:50.403378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.402965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"bbb7f48b014bb47eb4e9718f7d94b93d5b4e2ac7ab587bfa6410a8a6a9d1bda3"}
Jul 16 22:19:50.403378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.402990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"b92fb2b99eb3e58407cd23193dc1aa781aab7ad04a9806cab46aca793e6572b7"}
Jul 16 22:19:50.403378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.403016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"f7d7a134a5bb1ef71bd3763c4390a0fde4c4596d4ac709e6f1a8dd90135d8fcf"}
Jul 16 22:19:50.403378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.403040    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"2691573c406ee1fffd2fc882bdb7dbacc95d2c0d9dca3fd8fda335de5b1a763d"}
Jul 16 22:19:50.403378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.403062    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f48b920-277d-4cb9-a2bf-53f7dea480c4","Type":"ContainerStarted","Data":"e79d27d2448d1b7d12baba5d3b81f8765fc4384e38751a22f8c4541e78c8f0ba"}
Jul 16 22:19:50.429148 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:50.429060    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.413165779 podStartE2EDuration="2.429041593s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.875814ms" isStatefulPod=false podCreationTimestamp="2026-07-16 22:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:19:50.428505806 +0000 UTC m=+146.749061878" watchObservedRunningTime="2026-07-16 22:19:50.429041593 +0000 UTC m=+146.749597667"
Jul 16 22:19:51.885518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.885481    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c87dbfd89-lcp6f"]
Jul 16 22:19:51.887728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.887710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.901695 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.901671    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c87dbfd89-lcp6f"]
Jul 16 22:19:51.975358 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkcvc\" (UniqueName: \"kubernetes.io/projected/ff717ef8-79aa-4827-b7c1-9506bf330e6d-kube-api-access-pkcvc\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.975529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-trusted-ca-bundle\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.975529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-service-ca\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.975529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975461    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-serving-cert\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.975529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975477    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-config\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.975658 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-oauth-config\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:51.975658 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:51.975563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-oauth-serving-cert\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.076586 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-trusted-ca-bundle\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.076586 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-service-ca\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.076822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076609    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-serving-cert\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.076822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076729    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-config\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.076822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076806    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-oauth-config\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.077037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-oauth-serving-cert\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.077037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.076913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkcvc\" (UniqueName: \"kubernetes.io/projected/ff717ef8-79aa-4827-b7c1-9506bf330e6d-kube-api-access-pkcvc\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.077410 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.077383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-service-ca\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.077531 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.077514    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-trusted-ca-bundle\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.077611 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.077590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-config\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.077670 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.077617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-oauth-serving-cert\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.079280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.079262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-serving-cert\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.079368 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.079330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-oauth-config\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.089181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.089155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkcvc\" (UniqueName: \"kubernetes.io/projected/ff717ef8-79aa-4827-b7c1-9506bf330e6d-kube-api-access-pkcvc\") pod \"console-7c87dbfd89-lcp6f\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") " pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.196346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.196311    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:19:52.324727 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.324700    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c87dbfd89-lcp6f"]
Jul 16 22:19:52.326861 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:19:52.326825    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff717ef8_79aa_4827_b7c1_9506bf330e6d.slice/crio-f073c9b09707c90eab94da035f4cf20673130b0ea830d14f34106b85cec69b3b WatchSource:0}: Error finding container f073c9b09707c90eab94da035f4cf20673130b0ea830d14f34106b85cec69b3b: Status 404 returned error can't find the container with id f073c9b09707c90eab94da035f4cf20673130b0ea830d14f34106b85cec69b3b
Jul 16 22:19:52.412874 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.412840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87dbfd89-lcp6f" event={"ID":"ff717ef8-79aa-4827-b7c1-9506bf330e6d","Type":"ContainerStarted","Data":"53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a"}
Jul 16 22:19:52.413053 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.412888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87dbfd89-lcp6f" event={"ID":"ff717ef8-79aa-4827-b7c1-9506bf330e6d","Type":"ContainerStarted","Data":"f073c9b09707c90eab94da035f4cf20673130b0ea830d14f34106b85cec69b3b"}
Jul 16 22:19:52.443100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:19:52.443026    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c87dbfd89-lcp6f" podStartSLOduration=1.443008305 podStartE2EDuration="1.443008305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:19:52.442671308 +0000 UTC m=+148.763227382" watchObservedRunningTime="2026-07-16 22:19:52.443008305 +0000 UTC m=+148.763564376"
Jul 16 22:20:02.196670 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:02.196619    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:20:02.197236 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:02.196703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:20:02.201935 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:02.201907    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:20:02.453268 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:02.453192    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:20:02.499146 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:02.499103    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75c8864975-nmprh"]
Jul 16 22:20:19.929997 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.929914    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"]
Jul 16 22:20:19.932451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.932435    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:19.934755 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.934731    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 22:20:19.934755 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.934746    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 22:20:19.935465 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.935448    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-52rk8\""
Jul 16 22:20:19.941281 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.941258    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"]
Jul 16 22:20:19.991792 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.991751    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:19.991984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.991809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:19.991984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:19.991940    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtn68\" (UniqueName: \"kubernetes.io/projected/17d1ad06-57aa-483d-a518-d9df123a8d65-kube-api-access-qtn68\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.092787 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.092755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtn68\" (UniqueName: \"kubernetes.io/projected/17d1ad06-57aa-483d-a518-d9df123a8d65-kube-api-access-qtn68\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.092942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.092809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.092942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.092842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.093279 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.093261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.093360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.093298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.101403 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.101382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtn68\" (UniqueName: \"kubernetes.io/projected/17d1ad06-57aa-483d-a518-d9df123a8d65-kube-api-access-qtn68\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.242875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.242843    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:20.372678 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.372652    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"]
Jul 16 22:20:20.514980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:20.514896    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4" event={"ID":"17d1ad06-57aa-483d-a518-d9df123a8d65","Type":"ContainerStarted","Data":"96dbc1770f1785fbbd409d193d9d05cd66855faed905492a0a81f8bc66570d24"}
Jul 16 22:20:27.523558 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.523490    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75c8864975-nmprh" podUID="698acefb-c493-4a94-9156-dcd531013569" containerName="console" containerID="cri-o://9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03" gracePeriod=15
Jul 16 22:20:27.544768 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.544738    2564 generic.go:356] "Generic (PLEG): container finished" podID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerID="25f84027e3cc5e1705aa147de762af53ec4ea41524c4df7968ceaa68cd93c7b5" exitCode=0
Jul 16 22:20:27.544888 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.544790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4" event={"ID":"17d1ad06-57aa-483d-a518-d9df123a8d65","Type":"ContainerDied","Data":"25f84027e3cc5e1705aa147de762af53ec4ea41524c4df7968ceaa68cd93c7b5"}
Jul 16 22:20:27.774495 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.774438    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:20:27.857436 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857409    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-console-config\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857436 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857444    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-service-ca\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857519    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-oauth-config\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857545    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-trusted-ca-bundle\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857758 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857676    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-serving-cert\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857758 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857712    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/698acefb-c493-4a94-9156-dcd531013569-kube-api-access-nv8q7\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857758 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857737    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-oauth-serving-cert\") pod \"698acefb-c493-4a94-9156-dcd531013569\" (UID: \"698acefb-c493-4a94-9156-dcd531013569\") "
Jul 16 22:20:27.857903 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.857859    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-console-config" (OuterVolumeSpecName: "console-config") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:27.858170 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.858012    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-service-ca" (OuterVolumeSpecName: "service-ca") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:27.858170 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.858021    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-console-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:27.858170 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.858109    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:27.858360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.858254    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:27.859702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.859678    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:27.859779 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.859757    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698acefb-c493-4a94-9156-dcd531013569-kube-api-access-nv8q7" (OuterVolumeSpecName: "kube-api-access-nv8q7") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "kube-api-access-nv8q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:27.859824 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.859755    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "698acefb-c493-4a94-9156-dcd531013569" (UID: "698acefb-c493-4a94-9156-dcd531013569"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:27.959311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.959272    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-oauth-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:27.959311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.959304    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:27.959311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.959318    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-oauth-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:27.959548 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.959329    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/698acefb-c493-4a94-9156-dcd531013569-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:27.959548 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.959342    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/698acefb-c493-4a94-9156-dcd531013569-console-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:27.959548 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:27.959353    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/698acefb-c493-4a94-9156-dcd531013569-kube-api-access-nv8q7\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:28.550655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.550617    2564 generic.go:356] "Generic (PLEG): container finished" podID="698acefb-c493-4a94-9156-dcd531013569" containerID="9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03" exitCode=0
Jul 16 22:20:28.551109 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.550685    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c8864975-nmprh"
Jul 16 22:20:28.551109 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.550724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c8864975-nmprh" event={"ID":"698acefb-c493-4a94-9156-dcd531013569","Type":"ContainerDied","Data":"9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03"}
Jul 16 22:20:28.551109 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.550786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c8864975-nmprh" event={"ID":"698acefb-c493-4a94-9156-dcd531013569","Type":"ContainerDied","Data":"173b8b75f892ce4816bf7d5dca6f2c5d938ccd4dd05f0f5ae997793d4de72eed"}
Jul 16 22:20:28.551109 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.550813    2564 scope.go:117] "RemoveContainer" containerID="9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03"
Jul 16 22:20:28.560436 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.560402    2564 scope.go:117] "RemoveContainer" containerID="9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03"
Jul 16 22:20:28.560701 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:20:28.560670    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03\": container with ID starting with 9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03 not found: ID does not exist" containerID="9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03"
Jul 16 22:20:28.560779 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.560710    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03"} err="failed to get container status \"9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03\": rpc error: code = NotFound desc = could not find container \"9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03\": container with ID starting with 9726ca5c866758a89acc31426be16e238a8d411a908f3bc896d4cf8072288c03 not found: ID does not exist"
Jul 16 22:20:28.573846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.573817    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75c8864975-nmprh"]
Jul 16 22:20:28.579683 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:28.579652    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75c8864975-nmprh"]
Jul 16 22:20:29.601575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:29.601536    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="698acefb-c493-4a94-9156-dcd531013569" path="/var/lib/kubelet/pods/698acefb-c493-4a94-9156-dcd531013569/volumes"
Jul 16 22:20:30.560843 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:30.560809    2564 generic.go:356] "Generic (PLEG): container finished" podID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerID="2a7194230b48fc4aa7c4e18aa9e1c8bd752ce298c0314163d46d519a54b20a31" exitCode=0
Jul 16 22:20:30.561013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:30.560857    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4" event={"ID":"17d1ad06-57aa-483d-a518-d9df123a8d65","Type":"ContainerDied","Data":"2a7194230b48fc4aa7c4e18aa9e1c8bd752ce298c0314163d46d519a54b20a31"}
Jul 16 22:20:38.594863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:38.594824    2564 generic.go:356] "Generic (PLEG): container finished" podID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerID="a9288d64e5df88028783c8dca6e4863df9231204579e39caf75722acb431e6e5" exitCode=0
Jul 16 22:20:38.595264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:38.594866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4" event={"ID":"17d1ad06-57aa-483d-a518-d9df123a8d65","Type":"ContainerDied","Data":"a9288d64e5df88028783c8dca6e4863df9231204579e39caf75722acb431e6e5"}
Jul 16 22:20:39.728726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.728703    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:39.859744 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.859661    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtn68\" (UniqueName: \"kubernetes.io/projected/17d1ad06-57aa-483d-a518-d9df123a8d65-kube-api-access-qtn68\") pod \"17d1ad06-57aa-483d-a518-d9df123a8d65\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") "
Jul 16 22:20:39.859744 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.859733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-util\") pod \"17d1ad06-57aa-483d-a518-d9df123a8d65\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") "
Jul 16 22:20:39.859925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.859774    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-bundle\") pod \"17d1ad06-57aa-483d-a518-d9df123a8d65\" (UID: \"17d1ad06-57aa-483d-a518-d9df123a8d65\") "
Jul 16 22:20:39.860465 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.860435    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-bundle" (OuterVolumeSpecName: "bundle") pod "17d1ad06-57aa-483d-a518-d9df123a8d65" (UID: "17d1ad06-57aa-483d-a518-d9df123a8d65"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:39.861903 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.861871    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17d1ad06-57aa-483d-a518-d9df123a8d65-kube-api-access-qtn68" (OuterVolumeSpecName: "kube-api-access-qtn68") pod "17d1ad06-57aa-483d-a518-d9df123a8d65" (UID: "17d1ad06-57aa-483d-a518-d9df123a8d65"). InnerVolumeSpecName "kube-api-access-qtn68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:39.865839 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.865808    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-util" (OuterVolumeSpecName: "util") pod "17d1ad06-57aa-483d-a518-d9df123a8d65" (UID: "17d1ad06-57aa-483d-a518-d9df123a8d65"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:39.960333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.960299    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtn68\" (UniqueName: \"kubernetes.io/projected/17d1ad06-57aa-483d-a518-d9df123a8d65-kube-api-access-qtn68\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:39.960333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.960329    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:39.960333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:39.960340    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17d1ad06-57aa-483d-a518-d9df123a8d65-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:40.604895 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:40.604870    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4"
Jul 16 22:20:40.605050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:40.604856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408sln4" event={"ID":"17d1ad06-57aa-483d-a518-d9df123a8d65","Type":"ContainerDied","Data":"96dbc1770f1785fbbd409d193d9d05cd66855faed905492a0a81f8bc66570d24"}
Jul 16 22:20:40.605050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:40.604990    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96dbc1770f1785fbbd409d193d9d05cd66855faed905492a0a81f8bc66570d24"
Jul 16 22:20:48.153894 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.153852    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"]
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154256    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="extract"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154276    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="extract"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154288    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="pull"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154296    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="pull"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154318    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="util"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154326    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="util"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154350    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="698acefb-c493-4a94-9156-dcd531013569" containerName="console"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154358    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="698acefb-c493-4a94-9156-dcd531013569" containerName="console"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154417    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="698acefb-c493-4a94-9156-dcd531013569" containerName="console"
Jul 16 22:20:48.154496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.154431    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17d1ad06-57aa-483d-a518-d9df123a8d65" containerName="extract"
Jul 16 22:20:48.156349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.156328    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.159349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.159299    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 16 22:20:48.159480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.159426    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:20:48.159480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.159464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vklgr\""
Jul 16 22:20:48.171709 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.171687    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"]
Jul 16 22:20:48.228649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.228615    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrd7q\" (UniqueName: \"kubernetes.io/projected/2ed5c834-ea06-4798-b62b-51089ef91738-kube-api-access-xrd7q\") pod \"cert-manager-operator-controller-manager-55c444bf86-9sxh5\" (UID: \"2ed5c834-ea06-4798-b62b-51089ef91738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.228818 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.228703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c834-ea06-4798-b62b-51089ef91738-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9sxh5\" (UID: \"2ed5c834-ea06-4798-b62b-51089ef91738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.329492 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.329461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrd7q\" (UniqueName: \"kubernetes.io/projected/2ed5c834-ea06-4798-b62b-51089ef91738-kube-api-access-xrd7q\") pod \"cert-manager-operator-controller-manager-55c444bf86-9sxh5\" (UID: \"2ed5c834-ea06-4798-b62b-51089ef91738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.329675 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.329517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c834-ea06-4798-b62b-51089ef91738-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9sxh5\" (UID: \"2ed5c834-ea06-4798-b62b-51089ef91738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.329826 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.329800    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c834-ea06-4798-b62b-51089ef91738-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9sxh5\" (UID: \"2ed5c834-ea06-4798-b62b-51089ef91738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.337642 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.337621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrd7q\" (UniqueName: \"kubernetes.io/projected/2ed5c834-ea06-4798-b62b-51089ef91738-kube-api-access-xrd7q\") pod \"cert-manager-operator-controller-manager-55c444bf86-9sxh5\" (UID: \"2ed5c834-ea06-4798-b62b-51089ef91738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.465370 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.465340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"
Jul 16 22:20:48.595514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.595489    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5"]
Jul 16 22:20:48.597875 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:20:48.597834    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed5c834_ea06_4798_b62b_51089ef91738.slice/crio-8ad0a997f822dbc92f768424f8acc51e58393a5623c9d0773929878d8ada70e0 WatchSource:0}: Error finding container 8ad0a997f822dbc92f768424f8acc51e58393a5623c9d0773929878d8ada70e0: Status 404 returned error can't find the container with id 8ad0a997f822dbc92f768424f8acc51e58393a5623c9d0773929878d8ada70e0
Jul 16 22:20:48.636028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:48.635993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5" event={"ID":"2ed5c834-ea06-4798-b62b-51089ef91738","Type":"ContainerStarted","Data":"8ad0a997f822dbc92f768424f8acc51e58393a5623c9d0773929878d8ada70e0"}
Jul 16 22:20:50.648524 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:50.648403    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5" event={"ID":"2ed5c834-ea06-4798-b62b-51089ef91738","Type":"ContainerStarted","Data":"99bf6071633d032ec83c7258ff4adc2139c096197efcfee6365c57d8c4740c5e"}
Jul 16 22:20:50.671260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:50.671207    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9sxh5" podStartSLOduration=0.761679318 podStartE2EDuration="2.67118902s" totalImagesPullingTime="1.909509702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:20:50.669530192 +0000 UTC m=+206.990086264" watchObservedRunningTime="2026-07-16 22:20:50.67118902 +0000 UTC m=+206.991745092"
Jul 16 22:20:51.869502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.869468    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"]
Jul 16 22:20:51.871897 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.871879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:51.874484 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.874459    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 22:20:51.874606 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.874503    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 22:20:51.875123 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.875108    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-52rk8\""
Jul 16 22:20:51.883037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.883011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"]
Jul 16 22:20:51.956719 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.956683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx578\" (UniqueName: \"kubernetes.io/projected/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-kube-api-access-dx578\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:51.956891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.956780    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:51.956891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:51.956838    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.058129 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.058097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.058294 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.058182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.058294 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.058248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx578\" (UniqueName: \"kubernetes.io/projected/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-kube-api-access-dx578\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.058602 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.058578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.058602 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.058591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.067055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.067027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx578\" (UniqueName: \"kubernetes.io/projected/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-kube-api-access-dx578\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.180822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.180734    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:52.325374 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.325214    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"]
Jul 16 22:20:52.327883 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:20:52.327851    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f4c234_6dc9_4ed7_b017_65ed800b6e05.slice/crio-8df075839ba9b311ad2002fe33fc613d6f102362babaa68be0a09e2f30e026d5 WatchSource:0}: Error finding container 8df075839ba9b311ad2002fe33fc613d6f102362babaa68be0a09e2f30e026d5: Status 404 returned error can't find the container with id 8df075839ba9b311ad2002fe33fc613d6f102362babaa68be0a09e2f30e026d5
Jul 16 22:20:52.487298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.487264    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ppdfs"]
Jul 16 22:20:52.489699 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.489682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.492278 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.492258    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-m2hvl\""
Jul 16 22:20:52.492408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.492303    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 16 22:20:52.492567 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.492553    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 16 22:20:52.501251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.501228    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ppdfs"]
Jul 16 22:20:52.561872 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.561837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68c7fbb2-914e-4fed-a57a-f0f45f591c83-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ppdfs\" (UID: \"68c7fbb2-914e-4fed-a57a-f0f45f591c83\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.562039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.561894    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tbtm\" (UniqueName: \"kubernetes.io/projected/68c7fbb2-914e-4fed-a57a-f0f45f591c83-kube-api-access-9tbtm\") pod \"cert-manager-webhook-6486ffd997-ppdfs\" (UID: \"68c7fbb2-914e-4fed-a57a-f0f45f591c83\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.658012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.657981    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerID="89613b09fe50deb733d38ed8124b6637223f2ae193b3ea48f8c63eec8d9ab5e7" exitCode=0
Jul 16 22:20:52.658210 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.658068    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x" event={"ID":"e6f4c234-6dc9-4ed7-b017-65ed800b6e05","Type":"ContainerDied","Data":"89613b09fe50deb733d38ed8124b6637223f2ae193b3ea48f8c63eec8d9ab5e7"}
Jul 16 22:20:52.658210 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.658114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x" event={"ID":"e6f4c234-6dc9-4ed7-b017-65ed800b6e05","Type":"ContainerStarted","Data":"8df075839ba9b311ad2002fe33fc613d6f102362babaa68be0a09e2f30e026d5"}
Jul 16 22:20:52.662501 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.662474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68c7fbb2-914e-4fed-a57a-f0f45f591c83-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ppdfs\" (UID: \"68c7fbb2-914e-4fed-a57a-f0f45f591c83\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.662616 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.662602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tbtm\" (UniqueName: \"kubernetes.io/projected/68c7fbb2-914e-4fed-a57a-f0f45f591c83-kube-api-access-9tbtm\") pod \"cert-manager-webhook-6486ffd997-ppdfs\" (UID: \"68c7fbb2-914e-4fed-a57a-f0f45f591c83\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.670374 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.670350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68c7fbb2-914e-4fed-a57a-f0f45f591c83-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ppdfs\" (UID: \"68c7fbb2-914e-4fed-a57a-f0f45f591c83\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.670510 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.670492    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tbtm\" (UniqueName: \"kubernetes.io/projected/68c7fbb2-914e-4fed-a57a-f0f45f591c83-kube-api-access-9tbtm\") pod \"cert-manager-webhook-6486ffd997-ppdfs\" (UID: \"68c7fbb2-914e-4fed-a57a-f0f45f591c83\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.807104 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.807065    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:52.935289 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:52.935262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ppdfs"]
Jul 16 22:20:52.937601 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:20:52.937573    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68c7fbb2_914e_4fed_a57a_f0f45f591c83.slice/crio-5330299699ebef1446786ca69df96fa42fb2a176c05103d1c07909920407094c WatchSource:0}: Error finding container 5330299699ebef1446786ca69df96fa42fb2a176c05103d1c07909920407094c: Status 404 returned error can't find the container with id 5330299699ebef1446786ca69df96fa42fb2a176c05103d1c07909920407094c
Jul 16 22:20:53.665116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:53.665080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs" event={"ID":"68c7fbb2-914e-4fed-a57a-f0f45f591c83","Type":"ContainerStarted","Data":"5330299699ebef1446786ca69df96fa42fb2a176c05103d1c07909920407094c"}
Jul 16 22:20:54.881114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:54.881045    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"]
Jul 16 22:20:54.889111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:54.889079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:54.892160 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:54.892105    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xfdhd\""
Jul 16 22:20:54.895995 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:54.895935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"]
Jul 16 22:20:54.984020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:54.983856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42vk8\" (UniqueName: \"kubernetes.io/projected/db37ca8d-0e93-40a2-a367-51520e5225d8-kube-api-access-42vk8\") pod \"cert-manager-cainjector-559cdd8c87-4g9lr\" (UID: \"db37ca8d-0e93-40a2-a367-51520e5225d8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:54.984020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:54.983927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db37ca8d-0e93-40a2-a367-51520e5225d8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4g9lr\" (UID: \"db37ca8d-0e93-40a2-a367-51520e5225d8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:55.085281 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:55.085247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42vk8\" (UniqueName: \"kubernetes.io/projected/db37ca8d-0e93-40a2-a367-51520e5225d8-kube-api-access-42vk8\") pod \"cert-manager-cainjector-559cdd8c87-4g9lr\" (UID: \"db37ca8d-0e93-40a2-a367-51520e5225d8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:55.085467 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:55.085305    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db37ca8d-0e93-40a2-a367-51520e5225d8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4g9lr\" (UID: \"db37ca8d-0e93-40a2-a367-51520e5225d8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:55.094303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:55.094273    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db37ca8d-0e93-40a2-a367-51520e5225d8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4g9lr\" (UID: \"db37ca8d-0e93-40a2-a367-51520e5225d8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:55.094550 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:55.094436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42vk8\" (UniqueName: \"kubernetes.io/projected/db37ca8d-0e93-40a2-a367-51520e5225d8-kube-api-access-42vk8\") pod \"cert-manager-cainjector-559cdd8c87-4g9lr\" (UID: \"db37ca8d-0e93-40a2-a367-51520e5225d8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:55.202053 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:55.202022    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"
Jul 16 22:20:55.512482 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:55.512459    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr"]
Jul 16 22:20:56.303762 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:20:56.303719    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb37ca8d_0e93_40a2_a367_51520e5225d8.slice/crio-e4c0c742e7c981731dbea8cd630fe9606873c0e33bb56edae7c1e0c22fc5fb14 WatchSource:0}: Error finding container e4c0c742e7c981731dbea8cd630fe9606873c0e33bb56edae7c1e0c22fc5fb14: Status 404 returned error can't find the container with id e4c0c742e7c981731dbea8cd630fe9606873c0e33bb56edae7c1e0c22fc5fb14
Jul 16 22:20:56.679691 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:56.679600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr" event={"ID":"db37ca8d-0e93-40a2-a367-51520e5225d8","Type":"ContainerStarted","Data":"e4c0c742e7c981731dbea8cd630fe9606873c0e33bb56edae7c1e0c22fc5fb14"}
Jul 16 22:20:56.680966 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:56.680941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs" event={"ID":"68c7fbb2-914e-4fed-a57a-f0f45f591c83","Type":"ContainerStarted","Data":"f9f0644ece9fae028b158e51592455be623779479bf6518be9c78174b44cba1f"}
Jul 16 22:20:56.681039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:56.681012    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:20:56.682696 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:56.682659    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerID="35e2fa388c20d3fdd0deeb977daa2a637862bfd50ac34ef74350326765b2c485" exitCode=0
Jul 16 22:20:56.682802 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:56.682715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x" event={"ID":"e6f4c234-6dc9-4ed7-b017-65ed800b6e05","Type":"ContainerDied","Data":"35e2fa388c20d3fdd0deeb977daa2a637862bfd50ac34ef74350326765b2c485"}
Jul 16 22:20:56.696753 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:56.696706    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs" podStartSLOduration=1.293529017 podStartE2EDuration="4.696689066s" totalImagesPullingTime="3.403160049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:20:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:20:56.695885259 +0000 UTC m=+213.016441337" watchObservedRunningTime="2026-07-16 22:20:56.696689066 +0000 UTC m=+213.017245139"
Jul 16 22:20:57.689951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:57.689863    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerID="3439703eb52cd0bfed44b29d8e60da9775263d771f5d195c6efa2556a2afb4f6" exitCode=0
Jul 16 22:20:57.690355 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:57.689945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x" event={"ID":"e6f4c234-6dc9-4ed7-b017-65ed800b6e05","Type":"ContainerDied","Data":"3439703eb52cd0bfed44b29d8e60da9775263d771f5d195c6efa2556a2afb4f6"}
Jul 16 22:20:57.691392 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:57.691368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr" event={"ID":"db37ca8d-0e93-40a2-a367-51520e5225d8","Type":"ContainerStarted","Data":"a056f22faf9355b9593aeff9497e84bc4482872dcac3660a97ec6eb43083f236"}
Jul 16 22:20:57.764223 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:57.764168    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4g9lr" podStartSLOduration=2.7796412889999997 podStartE2EDuration="3.764130283s" totalImagesPullingTime="984.488994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:20:57.759837162 +0000 UTC m=+214.080393234" watchObservedRunningTime="2026-07-16 22:20:57.764130283 +0000 UTC m=+214.084686355"
Jul 16 22:20:58.824623 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.824598    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:20:58.920667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.920635    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx578\" (UniqueName: \"kubernetes.io/projected/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-kube-api-access-dx578\") pod \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") "
Jul 16 22:20:58.920839 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.920745    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-bundle\") pod \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") "
Jul 16 22:20:58.920839 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.920778    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-util\") pod \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\" (UID: \"e6f4c234-6dc9-4ed7-b017-65ed800b6e05\") "
Jul 16 22:20:58.921228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.921192    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-bundle" (OuterVolumeSpecName: "bundle") pod "e6f4c234-6dc9-4ed7-b017-65ed800b6e05" (UID: "e6f4c234-6dc9-4ed7-b017-65ed800b6e05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:58.922893 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.922866    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-kube-api-access-dx578" (OuterVolumeSpecName: "kube-api-access-dx578") pod "e6f4c234-6dc9-4ed7-b017-65ed800b6e05" (UID: "e6f4c234-6dc9-4ed7-b017-65ed800b6e05"). InnerVolumeSpecName "kube-api-access-dx578". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:58.925123 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:58.925102    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-util" (OuterVolumeSpecName: "util") pod "e6f4c234-6dc9-4ed7-b017-65ed800b6e05" (UID: "e6f4c234-6dc9-4ed7-b017-65ed800b6e05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:59.021982 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:59.021875    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:59.021982 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:59.021920    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:59.021982 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:59.021933    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx578\" (UniqueName: \"kubernetes.io/projected/e6f4c234-6dc9-4ed7-b017-65ed800b6e05-kube-api-access-dx578\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:20:59.702069 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:59.702036    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x" event={"ID":"e6f4c234-6dc9-4ed7-b017-65ed800b6e05","Type":"ContainerDied","Data":"8df075839ba9b311ad2002fe33fc613d6f102362babaa68be0a09e2f30e026d5"}
Jul 16 22:20:59.702249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:59.702087    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8df075839ba9b311ad2002fe33fc613d6f102362babaa68be0a09e2f30e026d5"
Jul 16 22:20:59.702249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:20:59.702047    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9kx5x"
Jul 16 22:21:02.693813 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:02.693784    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ppdfs"
Jul 16 22:21:13.951058 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951024    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"]
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951318    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="pull"
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951329    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="pull"
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951338    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="extract"
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951344    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="extract"
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951355    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="util"
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951360    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="util"
Jul 16 22:21:13.951476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.951425    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f4c234-6dc9-4ed7-b017-65ed800b6e05" containerName="extract"
Jul 16 22:21:13.955673 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.955654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:13.957951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.957926    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 22:21:13.958102 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.958035    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-52rk8\""
Jul 16 22:21:13.958693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.958667    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 22:21:13.963498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:13.963478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"]
Jul 16 22:21:14.044907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.044863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqwjt\" (UniqueName: \"kubernetes.io/projected/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-kube-api-access-jqwjt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.045093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.044937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.045093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.044965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.145793 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.145753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqwjt\" (UniqueName: \"kubernetes.io/projected/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-kube-api-access-jqwjt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.145952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.145827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.145952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.145857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.146337 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.146317    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.146381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.146314    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.153862 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.153838    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqwjt\" (UniqueName: \"kubernetes.io/projected/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-kube-api-access-jqwjt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.265288 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.265214    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:14.392514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.392486    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"]
Jul 16 22:21:14.393931 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:21:14.393903    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c20414_0fe1_4c9d_b3b3_fe0a376f69a4.slice/crio-4d8fc1d0ee87957570e2bc0166191723ad90cd378ad87fe0c9e31a52e8241c69 WatchSource:0}: Error finding container 4d8fc1d0ee87957570e2bc0166191723ad90cd378ad87fe0c9e31a52e8241c69: Status 404 returned error can't find the container with id 4d8fc1d0ee87957570e2bc0166191723ad90cd378ad87fe0c9e31a52e8241c69
Jul 16 22:21:14.768331 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.768295    2564 generic.go:356] "Generic (PLEG): container finished" podID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerID="1352e579f4609f12e53982e48d9c4417df10e7bdfa6355fd3f166d18be59cb05" exitCode=0
Jul 16 22:21:14.768503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.768353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh" event={"ID":"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4","Type":"ContainerDied","Data":"1352e579f4609f12e53982e48d9c4417df10e7bdfa6355fd3f166d18be59cb05"}
Jul 16 22:21:14.768503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:14.768390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh" event={"ID":"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4","Type":"ContainerStarted","Data":"4d8fc1d0ee87957570e2bc0166191723ad90cd378ad87fe0c9e31a52e8241c69"}
Jul 16 22:21:15.774627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:15.774585    2564 generic.go:356] "Generic (PLEG): container finished" podID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerID="366b877a6eb48d848f57b93dd06076271f91829e94c6bd004ee6922d4906121c" exitCode=0
Jul 16 22:21:15.775119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:15.774675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh" event={"ID":"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4","Type":"ContainerDied","Data":"366b877a6eb48d848f57b93dd06076271f91829e94c6bd004ee6922d4906121c"}
Jul 16 22:21:16.780756 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:16.780722    2564 generic.go:356] "Generic (PLEG): container finished" podID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerID="3ad4233018115bfdb36b8bf2062d86b3116d5fbf899b86bc50b86a7b0e03c1e6" exitCode=0
Jul 16 22:21:16.781122 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:16.780807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh" event={"ID":"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4","Type":"ContainerDied","Data":"3ad4233018115bfdb36b8bf2062d86b3116d5fbf899b86bc50b86a7b0e03c1e6"}
Jul 16 22:21:17.911722 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.911696    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:17.976564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.976529    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-util\") pod \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") "
Jul 16 22:21:17.976748 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.976573    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqwjt\" (UniqueName: \"kubernetes.io/projected/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-kube-api-access-jqwjt\") pod \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") "
Jul 16 22:21:17.976748 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.976648    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-bundle\") pod \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\" (UID: \"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4\") "
Jul 16 22:21:17.977401 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.977375    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-bundle" (OuterVolumeSpecName: "bundle") pod "87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" (UID: "87c20414-0fe1-4c9d-b3b3-fe0a376f69a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:21:17.978658 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.978630    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-kube-api-access-jqwjt" (OuterVolumeSpecName: "kube-api-access-jqwjt") pod "87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" (UID: "87c20414-0fe1-4c9d-b3b3-fe0a376f69a4"). InnerVolumeSpecName "kube-api-access-jqwjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:21:17.982555 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:17.982530    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-util" (OuterVolumeSpecName: "util") pod "87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" (UID: "87c20414-0fe1-4c9d-b3b3-fe0a376f69a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:21:18.077877 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:18.077790    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:21:18.077877 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:18.077818    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqwjt\" (UniqueName: \"kubernetes.io/projected/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-kube-api-access-jqwjt\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:21:18.077877 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:18.077828    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c20414-0fe1-4c9d-b3b3-fe0a376f69a4-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:21:18.790746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:18.790720    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh"
Jul 16 22:21:18.790746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:18.790729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwftkh" event={"ID":"87c20414-0fe1-4c9d-b3b3-fe0a376f69a4","Type":"ContainerDied","Data":"4d8fc1d0ee87957570e2bc0166191723ad90cd378ad87fe0c9e31a52e8241c69"}
Jul 16 22:21:18.790993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:18.790770    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8fc1d0ee87957570e2bc0166191723ad90cd378ad87fe0c9e31a52e8241c69"
Jul 16 22:21:29.312853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.312818    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"]
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="extract"
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313117    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="extract"
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313129    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="util"
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313148    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="util"
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313158    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="pull"
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313164    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="pull"
Jul 16 22:21:29.313262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.313215    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c20414-0fe1-4c9d-b3b3-fe0a376f69a4" containerName="extract"
Jul 16 22:21:29.316088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.316072    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.319272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.319251    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 22:21:29.319968 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.319946    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-592w7\""
Jul 16 22:21:29.320240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.320216    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 22:21:29.320240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.320235    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 22:21:29.320541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.320524    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 22:21:29.333891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.333866    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"]
Jul 16 22:21:29.372536 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.372497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.372684 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.372541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7k8\" (UniqueName: \"kubernetes.io/projected/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-kube-api-access-tg7k8\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.372684 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.372578    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-manifests\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.372684 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.372597    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-tmp\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.372684 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.372665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.473354 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.473313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-manifests\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.473354 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.473352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-tmp\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.473585 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.473385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.473585 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.473439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.473585 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.473489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7k8\" (UniqueName: \"kubernetes.io/projected/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-kube-api-access-tg7k8\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.473798 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.473774    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-manifests\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.475900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.475865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-tmp\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.476008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.475989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.476073 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.476054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.487931 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.487904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7k8\" (UniqueName: \"kubernetes.io/projected/f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928-kube-api-access-tg7k8\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt\" (UID: \"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.626356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.626263    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:29.794935 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.794910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"]
Jul 16 22:21:29.796503 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:21:29.796475    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf67795cc_3b7b_4ccb_8aa7_80a5b5cbf928.slice/crio-2c7cf2f4389e897ec81aa8418977306bbcb7ae2c0ef05270a8ccf9b78334b3c4 WatchSource:0}: Error finding container 2c7cf2f4389e897ec81aa8418977306bbcb7ae2c0ef05270a8ccf9b78334b3c4: Status 404 returned error can't find the container with id 2c7cf2f4389e897ec81aa8418977306bbcb7ae2c0ef05270a8ccf9b78334b3c4
Jul 16 22:21:29.841790 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:29.841747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt" event={"ID":"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928","Type":"ContainerStarted","Data":"2c7cf2f4389e897ec81aa8418977306bbcb7ae2c0ef05270a8ccf9b78334b3c4"}
Jul 16 22:21:33.860808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:33.860770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt" event={"ID":"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928","Type":"ContainerStarted","Data":"7dcf9f6dfb8430feca9b1f891bb237021918768b563b24417200a7915d9fcf68"}
Jul 16 22:21:34.866114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:34.866075    2564 generic.go:356] "Generic (PLEG): container finished" podID="f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928" containerID="7dcf9f6dfb8430feca9b1f891bb237021918768b563b24417200a7915d9fcf68" exitCode=0
Jul 16 22:21:34.866604 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:34.866159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt" event={"ID":"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928","Type":"ContainerDied","Data":"7dcf9f6dfb8430feca9b1f891bb237021918768b563b24417200a7915d9fcf68"}
Jul 16 22:21:35.872909 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:35.872870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt" event={"ID":"f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928","Type":"ContainerStarted","Data":"dc589ba4595811c4108ebbf1d35875e83d50d4cd951b02fe2f5fd1fdcfabdcaf"}
Jul 16 22:21:35.873292 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:35.872968    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:35.896860 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:35.896806    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt" podStartSLOduration=2.2500087349999998 podStartE2EDuration="6.896788714s" totalImagesPullingTime="3.591875386s" totalInitContainerRuntime="1.054904593s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:35.894235385 +0000 UTC m=+252.214791456" watchObservedRunningTime="2026-07-16 22:21:35.896788714 +0000 UTC m=+252.217344786"
Jul 16 22:21:46.879417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.879341    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt"
Jul 16 22:21:46.966326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.966292    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"]
Jul 16 22:21:46.972676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.972651    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:46.975122 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.975099    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 22:21:46.975286 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.975179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 22:21:46.975355 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.975285    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xzcpf\""
Jul 16 22:21:46.975355 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.975344    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 22:21:46.975459 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.975348    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 22:21:46.975507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.975461    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:21:46.979556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:46.979531    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"]
Jul 16 22:21:47.128574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.128538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cc547ee2-89a9-47ab-ab66-1132977e07a8-manager-config\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.128751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.128592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc547ee2-89a9-47ab-ab66-1132977e07a8-cert\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.128751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.128632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhbf\" (UniqueName: \"kubernetes.io/projected/cc547ee2-89a9-47ab-ab66-1132977e07a8-kube-api-access-9xhbf\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.128751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.128677    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc547ee2-89a9-47ab-ab66-1132977e07a8-metrics-cert\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.229669 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.229632    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc547ee2-89a9-47ab-ab66-1132977e07a8-cert\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.229852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.229676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhbf\" (UniqueName: \"kubernetes.io/projected/cc547ee2-89a9-47ab-ab66-1132977e07a8-kube-api-access-9xhbf\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.229852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.229711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc547ee2-89a9-47ab-ab66-1132977e07a8-metrics-cert\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.229852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.229759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cc547ee2-89a9-47ab-ab66-1132977e07a8-manager-config\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.230452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.230427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cc547ee2-89a9-47ab-ab66-1132977e07a8-manager-config\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.232149 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.232108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc547ee2-89a9-47ab-ab66-1132977e07a8-metrics-cert\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.232236 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.232177    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc547ee2-89a9-47ab-ab66-1132977e07a8-cert\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.241893 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.241870    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhbf\" (UniqueName: \"kubernetes.io/projected/cc547ee2-89a9-47ab-ab66-1132977e07a8-kube-api-access-9xhbf\") pod \"lws-controller-manager-67c6655946-gbvs7\" (UID: \"cc547ee2-89a9-47ab-ab66-1132977e07a8\") " pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.283042 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.283009    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:47.415383 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.415316    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"]
Jul 16 22:21:47.417392 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:21:47.417369    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc547ee2_89a9_47ab_ab66_1132977e07a8.slice/crio-1fd083edda2dda2c6b737a512b2ade2f0f8ecaf44007a34175d48d0a70af6ff0 WatchSource:0}: Error finding container 1fd083edda2dda2c6b737a512b2ade2f0f8ecaf44007a34175d48d0a70af6ff0: Status 404 returned error can't find the container with id 1fd083edda2dda2c6b737a512b2ade2f0f8ecaf44007a34175d48d0a70af6ff0
Jul 16 22:21:47.924752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:47.924719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7" event={"ID":"cc547ee2-89a9-47ab-ab66-1132977e07a8","Type":"ContainerStarted","Data":"1fd083edda2dda2c6b737a512b2ade2f0f8ecaf44007a34175d48d0a70af6ff0"}
Jul 16 22:21:49.935477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:49.935437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7" event={"ID":"cc547ee2-89a9-47ab-ab66-1132977e07a8","Type":"ContainerStarted","Data":"2818887be581d00f2a7b159711dd5e5df00f1e1c7b4cbc8d3d66b3c70cd8d046"}
Jul 16 22:21:49.935871 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:49.935518    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:21:49.957980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:49.957927    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7" podStartSLOduration=2.211580372 podStartE2EDuration="3.957908333s" totalImagesPullingTime="1.746327961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:49.956501823 +0000 UTC m=+266.277057896" watchObservedRunningTime="2026-07-16 22:21:49.957908333 +0000 UTC m=+266.278464405"
Jul 16 22:21:58.021905 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.021869    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"]
Jul 16 22:21:58.024271 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.024252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.027391 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.027371    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 22:21:58.027746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.027729    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p2v5l\""
Jul 16 22:21:58.027801 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.027754    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 22:21:58.027853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.027766    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 22:21:58.039609 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.039588    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"]
Jul 16 22:21:58.063320 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.063288    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"]
Jul 16 22:21:58.065364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.065348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.067814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.067791    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ksph6\""
Jul 16 22:21:58.067934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.067813    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 22:21:58.068023 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.068011    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 22:21:58.068241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.068226    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 22:21:58.085203 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.085167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"]
Jul 16 22:21:58.130399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130355    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a704999-554c-449e-8ed3-3490103090e8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130494    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr7vx\" (UniqueName: \"kubernetes.io/projected/5a704999-554c-449e-8ed3-3490103090e8-kube-api-access-vr7vx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a704999-554c-449e-8ed3-3490103090e8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130762 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a704999-554c-449e-8ed3-3490103090e8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130762 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130638    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.130762 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.130680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.231500 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a704999-554c-449e-8ed3-3490103090e8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.231500 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231506    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.231747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a704999-554c-449e-8ed3-3490103090e8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.231747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.231747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7785x\" (UniqueName: \"kubernetes.io/projected/1e9616a3-2269-4532-9c04-b7a8d6c88923-kube-api-access-7785x\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.231747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231642    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.231747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231670    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.231747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a704999-554c-449e-8ed3-3490103090e8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.231978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr7vx\" (UniqueName: \"kubernetes.io/projected/5a704999-554c-449e-8ed3-3490103090e8-kube-api-access-vr7vx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9616a3-2269-4532-9c04-b7a8d6c88923-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.232451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232016    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a704999-554c-449e-8ed3-3490103090e8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.232451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.232427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.233833 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.233807    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a704999-554c-449e-8ed3-3490103090e8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.234161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.234128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a704999-554c-449e-8ed3-3490103090e8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.240870 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.240848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a704999-554c-449e-8ed3-3490103090e8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.241022 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.240999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr7vx\" (UniqueName: \"kubernetes.io/projected/5a704999-554c-449e-8ed3-3490103090e8-kube-api-access-vr7vx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bc6qb\" (UID: \"5a704999-554c-449e-8ed3-3490103090e8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.332724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7785x\" (UniqueName: \"kubernetes.io/projected/1e9616a3-2269-4532-9c04-b7a8d6c88923-kube-api-access-7785x\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.332724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.332724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.332993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.332993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.332993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332797    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.332993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.332954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9616a3-2269-4532-9c04-b7a8d6c88923-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.333668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.333645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.334430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.334414    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:21:58.335115 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.335094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9616a3-2269-4532-9c04-b7a8d6c88923-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.335356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.335335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.335569 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.335550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.335611 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.335579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e9616a3-2269-4532-9c04-b7a8d6c88923-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.344215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.344187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9616a3-2269-4532-9c04-b7a8d6c88923-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.357033 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.357003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7785x\" (UniqueName: \"kubernetes.io/projected/1e9616a3-2269-4532-9c04-b7a8d6c88923-kube-api-access-7785x\") pod \"istiod-openshift-gateway-94bb8fbfd-v29lv\" (UID: \"1e9616a3-2269-4532-9c04-b7a8d6c88923\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.374543 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.374510    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:21:58.485084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.485045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"]
Jul 16 22:21:58.486966 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:21:58.486940    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a704999_554c_449e_8ed3_3490103090e8.slice/crio-996568ddf8ba323abe5f0781432eab5ac23236884f05016551c50716852dc4c3 WatchSource:0}: Error finding container 996568ddf8ba323abe5f0781432eab5ac23236884f05016551c50716852dc4c3: Status 404 returned error can't find the container with id 996568ddf8ba323abe5f0781432eab5ac23236884f05016551c50716852dc4c3
Jul 16 22:21:58.527543 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.527508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"]
Jul 16 22:21:58.531432 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:21:58.531405    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9616a3_2269_4532_9c04_b7a8d6c88923.slice/crio-7709aa09ebba4d05838c5656dbb3d5db3cbe80fbc758b70de51d65c4c3710d7d WatchSource:0}: Error finding container 7709aa09ebba4d05838c5656dbb3d5db3cbe80fbc758b70de51d65c4c3710d7d: Status 404 returned error can't find the container with id 7709aa09ebba4d05838c5656dbb3d5db3cbe80fbc758b70de51d65c4c3710d7d
Jul 16 22:21:58.978407 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.978356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb" event={"ID":"5a704999-554c-449e-8ed3-3490103090e8","Type":"ContainerStarted","Data":"996568ddf8ba323abe5f0781432eab5ac23236884f05016551c50716852dc4c3"}
Jul 16 22:21:58.979901 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:21:58.979864    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv" event={"ID":"1e9616a3-2269-4532-9c04-b7a8d6c88923","Type":"ContainerStarted","Data":"7709aa09ebba4d05838c5656dbb3d5db3cbe80fbc758b70de51d65c4c3710d7d"}
Jul 16 22:22:00.942604 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:00.942574    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67c6655946-gbvs7"
Jul 16 22:22:02.087814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:02.087767    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:22:02.088096 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:02.087856    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:22:02.088096 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:02.087767    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:22:02.088096 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:02.087881    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:22:02.088096 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:02.087924    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:22:03.004439 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.004380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv" event={"ID":"1e9616a3-2269-4532-9c04-b7a8d6c88923","Type":"ContainerStarted","Data":"9a1f72eda1944f4b1b043a4ab4cf10b4f9d0fac6eccddc6df31ef5081eb91f1d"}
Jul 16 22:22:03.004655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.004459    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:22:03.006090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.006057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb" event={"ID":"5a704999-554c-449e-8ed3-3490103090e8","Type":"ContainerStarted","Data":"c95192fca4830e6803ff275b7cd1fd1e099c2c4ad000f8dec9741fcd0d5e3129"}
Jul 16 22:22:03.006448 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.006430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv"
Jul 16 22:22:03.035712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.035658    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v29lv" podStartSLOduration=1.481412528 podStartE2EDuration="5.035639482s" totalImagesPullingTime="3.554226954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:03.033839013 +0000 UTC m=+279.354395086" watchObservedRunningTime="2026-07-16 22:22:03.035639482 +0000 UTC m=+279.356195554"
Jul 16 22:22:03.335659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.335566    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:22:03.340499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.340473    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:22:03.368837 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:03.368786    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb" podStartSLOduration=1.770538159 podStartE2EDuration="5.368769272s" totalImagesPullingTime="3.598231113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:03.101270748 +0000 UTC m=+279.421826819" watchObservedRunningTime="2026-07-16 22:22:03.368769272 +0000 UTC m=+279.689325343"
Jul 16 22:22:04.011272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:04.011234    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:22:04.012331 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:04.012308    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bc6qb"
Jul 16 22:22:24.302984 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:24.302952    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:22:24.305195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:24.305170    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:22:24.309627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:22:24.309608    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 22:23:05.505840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.505797    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"]
Jul 16 22:23:05.509265 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.509242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.511810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.511789    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r9mf7\""
Jul 16 22:23:05.512008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.511987    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 22:23:05.512578 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.512561    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 22:23:05.516159 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.516116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"]
Jul 16 22:23:05.611435 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.611401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.611628 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.611444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.611628 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.611572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffpgs\" (UniqueName: \"kubernetes.io/projected/d560b140-abf1-42b7-bbe9-b907ed8d856e-kube-api-access-ffpgs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.712081 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.712043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.712081 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.712089    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.712296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.712156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpgs\" (UniqueName: \"kubernetes.io/projected/d560b140-abf1-42b7-bbe9-b907ed8d856e-kube-api-access-ffpgs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.712559 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.712535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.712600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.712547    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.722901 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.722865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffpgs\" (UniqueName: \"kubernetes.io/projected/d560b140-abf1-42b7-bbe9-b907ed8d856e-kube-api-access-ffpgs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.820794 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.820694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:05.950939 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:05.950906    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"]
Jul 16 22:23:05.953908 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:23:05.953879    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd560b140_abf1_42b7_bbe9_b907ed8d856e.slice/crio-2dcd15f47e8562d57c375995503f3f9ac83e390eff0b32508360eb9d448566c6 WatchSource:0}: Error finding container 2dcd15f47e8562d57c375995503f3f9ac83e390eff0b32508360eb9d448566c6: Status 404 returned error can't find the container with id 2dcd15f47e8562d57c375995503f3f9ac83e390eff0b32508360eb9d448566c6
Jul 16 22:23:06.114012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.113938    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"]
Jul 16 22:23:06.116640 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.116623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.125808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.125783    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"]
Jul 16 22:23:06.215856 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.215806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.215856 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.215861    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lnz5\" (UniqueName: \"kubernetes.io/projected/073f84a5-5a1d-40c1-987e-771ef6d09f3d-kube-api-access-2lnz5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.216073 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.215952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.294806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.294773    2564 generic.go:356] "Generic (PLEG): container finished" podID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerID="7948ac3a0ed2dd260304e46930e14de0f1d7f7cc50041272960ce68dee2cabd8" exitCode=0
Jul 16 22:23:06.294960 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.294847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7" event={"ID":"d560b140-abf1-42b7-bbe9-b907ed8d856e","Type":"ContainerDied","Data":"7948ac3a0ed2dd260304e46930e14de0f1d7f7cc50041272960ce68dee2cabd8"}
Jul 16 22:23:06.294960 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.294887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7" event={"ID":"d560b140-abf1-42b7-bbe9-b907ed8d856e","Type":"ContainerStarted","Data":"2dcd15f47e8562d57c375995503f3f9ac83e390eff0b32508360eb9d448566c6"}
Jul 16 22:23:06.295377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.295361    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:23:06.316430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.316399    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.316581 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.316485    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.316581 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.316514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lnz5\" (UniqueName: \"kubernetes.io/projected/073f84a5-5a1d-40c1-987e-771ef6d09f3d-kube-api-access-2lnz5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.316892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.316871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.316937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.316859    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.325404 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.325374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lnz5\" (UniqueName: \"kubernetes.io/projected/073f84a5-5a1d-40c1-987e-771ef6d09f3d-kube-api-access-2lnz5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.426341 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.426268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:06.509875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.509850    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"]
Jul 16 22:23:06.513180 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.512871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.523396 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.523360    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"]
Jul 16 22:23:06.564531 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.564507    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"]
Jul 16 22:23:06.566821 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:23:06.566795    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073f84a5_5a1d_40c1_987e_771ef6d09f3d.slice/crio-35b8b6bf1649949e1cf6af497969647cd3a763bf92ed6a05849698c42cf6e637 WatchSource:0}: Error finding container 35b8b6bf1649949e1cf6af497969647cd3a763bf92ed6a05849698c42cf6e637: Status 404 returned error can't find the container with id 35b8b6bf1649949e1cf6af497969647cd3a763bf92ed6a05849698c42cf6e637
Jul 16 22:23:06.618784 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.618758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.618896 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.618806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnmkt\" (UniqueName: \"kubernetes.io/projected/aef2396c-fea2-4fdc-b364-14c63c900547-kube-api-access-cnmkt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.618969 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.618945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.719858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.719824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.720030 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.719891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.720030 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.719913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnmkt\" (UniqueName: \"kubernetes.io/projected/aef2396c-fea2-4fdc-b364-14c63c900547-kube-api-access-cnmkt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.720288 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.720267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.720333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.720303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.728647 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.728619    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnmkt\" (UniqueName: \"kubernetes.io/projected/aef2396c-fea2-4fdc-b364-14c63c900547-kube-api-access-cnmkt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.824916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.824876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:06.903318 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.903287    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"]
Jul 16 22:23:06.906162 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.906121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:06.922254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.919884    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"]
Jul 16 22:23:06.964778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:06.964747    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"]
Jul 16 22:23:06.967156 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:23:06.967103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef2396c_fea2_4fdc_b364_14c63c900547.slice/crio-32705af6fd3927a48dbc1f310b0b513b81d648d7da754e7f76f116aa579602d9 WatchSource:0}: Error finding container 32705af6fd3927a48dbc1f310b0b513b81d648d7da754e7f76f116aa579602d9: Status 404 returned error can't find the container with id 32705af6fd3927a48dbc1f310b0b513b81d648d7da754e7f76f116aa579602d9
Jul 16 22:23:07.025269 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.025243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.025410 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.025276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x74tw\" (UniqueName: \"kubernetes.io/projected/d616be2e-4d83-4d91-8640-76078db8c7a4-kube-api-access-x74tw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.025410 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.025373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.126813 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.126783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.126970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.126845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.126970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.126864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x74tw\" (UniqueName: \"kubernetes.io/projected/d616be2e-4d83-4d91-8640-76078db8c7a4-kube-api-access-x74tw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.127169 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.127126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.127213 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.127178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.135681 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.135661    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x74tw\" (UniqueName: \"kubernetes.io/projected/d616be2e-4d83-4d91-8640-76078db8c7a4-kube-api-access-x74tw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.219088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.219055    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:07.303440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.303408    2564 generic.go:356] "Generic (PLEG): container finished" podID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerID="8bcead6b0485e92112cf3ed6f64d5e65f804801af0bfd9d86644eb67d6293a45" exitCode=0
Jul 16 22:23:07.303593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.303544    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7" event={"ID":"d560b140-abf1-42b7-bbe9-b907ed8d856e","Type":"ContainerDied","Data":"8bcead6b0485e92112cf3ed6f64d5e65f804801af0bfd9d86644eb67d6293a45"}
Jul 16 22:23:07.305705 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.305675    2564 generic.go:356] "Generic (PLEG): container finished" podID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerID="7ecf9cf598873bca7eba131a4a0f4c6514cc3ab19fa6fcd088168222afe545e1" exitCode=0
Jul 16 22:23:07.305830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.305782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj" event={"ID":"073f84a5-5a1d-40c1-987e-771ef6d09f3d","Type":"ContainerDied","Data":"7ecf9cf598873bca7eba131a4a0f4c6514cc3ab19fa6fcd088168222afe545e1"}
Jul 16 22:23:07.305830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.305827    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj" event={"ID":"073f84a5-5a1d-40c1-987e-771ef6d09f3d","Type":"ContainerStarted","Data":"35b8b6bf1649949e1cf6af497969647cd3a763bf92ed6a05849698c42cf6e637"}
Jul 16 22:23:07.307985 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.307802    2564 generic.go:356] "Generic (PLEG): container finished" podID="aef2396c-fea2-4fdc-b364-14c63c900547" containerID="c095578db9019c16361b0475b27f94ec1a686b9d8b2b35b6c42ae6a930f3036f" exitCode=0
Jul 16 22:23:07.307985 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.307856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9" event={"ID":"aef2396c-fea2-4fdc-b364-14c63c900547","Type":"ContainerDied","Data":"c095578db9019c16361b0475b27f94ec1a686b9d8b2b35b6c42ae6a930f3036f"}
Jul 16 22:23:07.307985 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.307890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9" event={"ID":"aef2396c-fea2-4fdc-b364-14c63c900547","Type":"ContainerStarted","Data":"32705af6fd3927a48dbc1f310b0b513b81d648d7da754e7f76f116aa579602d9"}
Jul 16 22:23:07.356820 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:07.356792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"]
Jul 16 22:23:07.358373 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:23:07.358345    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd616be2e_4d83_4d91_8640_76078db8c7a4.slice/crio-a2288e546c0353db7785922efadc963cbe73c6b4049695354a55a9598b112470 WatchSource:0}: Error finding container a2288e546c0353db7785922efadc963cbe73c6b4049695354a55a9598b112470: Status 404 returned error can't find the container with id a2288e546c0353db7785922efadc963cbe73c6b4049695354a55a9598b112470
Jul 16 22:23:08.315110 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:08.315079    2564 generic.go:356] "Generic (PLEG): container finished" podID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerID="66788ceb640f49ece9ba8e1fb4725ced68546acdb23227a81408f30561ce4fca" exitCode=0
Jul 16 22:23:08.315499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:08.315175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7" event={"ID":"d560b140-abf1-42b7-bbe9-b907ed8d856e","Type":"ContainerDied","Data":"66788ceb640f49ece9ba8e1fb4725ced68546acdb23227a81408f30561ce4fca"}
Jul 16 22:23:08.316838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:08.316813    2564 generic.go:356] "Generic (PLEG): container finished" podID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerID="1f253c1b36a0a0a81b265428d56ac02fee0f77f04a75bfc7d85747eb38e4f931" exitCode=0
Jul 16 22:23:08.316956 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:08.316884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5" event={"ID":"d616be2e-4d83-4d91-8640-76078db8c7a4","Type":"ContainerDied","Data":"1f253c1b36a0a0a81b265428d56ac02fee0f77f04a75bfc7d85747eb38e4f931"}
Jul 16 22:23:08.316956 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:08.316940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5" event={"ID":"d616be2e-4d83-4d91-8640-76078db8c7a4","Type":"ContainerStarted","Data":"a2288e546c0353db7785922efadc963cbe73c6b4049695354a55a9598b112470"}
Jul 16 22:23:08.318987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:08.318961    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj" event={"ID":"073f84a5-5a1d-40c1-987e-771ef6d09f3d","Type":"ContainerStarted","Data":"be10f97f1469215e4b1093f9e8b0a08337a83b45f2706a007cf3fc9795003125"}
Jul 16 22:23:09.325722 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.325689    2564 generic.go:356] "Generic (PLEG): container finished" podID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerID="d2172a68d010ee98e98b76e1bcb7b25c6a98a563c1c917594916f5e100fe96af" exitCode=0
Jul 16 22:23:09.326120 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.325767    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5" event={"ID":"d616be2e-4d83-4d91-8640-76078db8c7a4","Type":"ContainerDied","Data":"d2172a68d010ee98e98b76e1bcb7b25c6a98a563c1c917594916f5e100fe96af"}
Jul 16 22:23:09.327685 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.327665    2564 generic.go:356] "Generic (PLEG): container finished" podID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerID="be10f97f1469215e4b1093f9e8b0a08337a83b45f2706a007cf3fc9795003125" exitCode=0
Jul 16 22:23:09.327777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.327709    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj" event={"ID":"073f84a5-5a1d-40c1-987e-771ef6d09f3d","Type":"ContainerDied","Data":"be10f97f1469215e4b1093f9e8b0a08337a83b45f2706a007cf3fc9795003125"}
Jul 16 22:23:09.329800 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.329769    2564 generic.go:356] "Generic (PLEG): container finished" podID="aef2396c-fea2-4fdc-b364-14c63c900547" containerID="c7f5abffc9968de5837014cee5ec41281a6df084fb4fedcfb2ab6d2a9524f78e" exitCode=0
Jul 16 22:23:09.329879 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.329841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9" event={"ID":"aef2396c-fea2-4fdc-b364-14c63c900547","Type":"ContainerDied","Data":"c7f5abffc9968de5837014cee5ec41281a6df084fb4fedcfb2ab6d2a9524f78e"}
Jul 16 22:23:09.477143 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.477103    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:09.547916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.547882    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffpgs\" (UniqueName: \"kubernetes.io/projected/d560b140-abf1-42b7-bbe9-b907ed8d856e-kube-api-access-ffpgs\") pod \"d560b140-abf1-42b7-bbe9-b907ed8d856e\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") "
Jul 16 22:23:09.548097 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.547954    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-bundle\") pod \"d560b140-abf1-42b7-bbe9-b907ed8d856e\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") "
Jul 16 22:23:09.548097 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.547979    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-util\") pod \"d560b140-abf1-42b7-bbe9-b907ed8d856e\" (UID: \"d560b140-abf1-42b7-bbe9-b907ed8d856e\") "
Jul 16 22:23:09.548421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.548388    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-bundle" (OuterVolumeSpecName: "bundle") pod "d560b140-abf1-42b7-bbe9-b907ed8d856e" (UID: "d560b140-abf1-42b7-bbe9-b907ed8d856e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:09.550091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.550068    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d560b140-abf1-42b7-bbe9-b907ed8d856e-kube-api-access-ffpgs" (OuterVolumeSpecName: "kube-api-access-ffpgs") pod "d560b140-abf1-42b7-bbe9-b907ed8d856e" (UID: "d560b140-abf1-42b7-bbe9-b907ed8d856e"). InnerVolumeSpecName "kube-api-access-ffpgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:23:09.553748 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.553722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-util" (OuterVolumeSpecName: "util") pod "d560b140-abf1-42b7-bbe9-b907ed8d856e" (UID: "d560b140-abf1-42b7-bbe9-b907ed8d856e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:09.649013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.648970    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffpgs\" (UniqueName: \"kubernetes.io/projected/d560b140-abf1-42b7-bbe9-b907ed8d856e-kube-api-access-ffpgs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:09.649013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.649008    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:09.649013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:09.649024    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d560b140-abf1-42b7-bbe9-b907ed8d856e-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:10.336611 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.336570    2564 generic.go:356] "Generic (PLEG): container finished" podID="aef2396c-fea2-4fdc-b364-14c63c900547" containerID="e9c5d4664e6a6af1a280adc82f0f7baad6cecdbafdfe9445a91959cb79c54639" exitCode=0
Jul 16 22:23:10.337010 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.336652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9" event={"ID":"aef2396c-fea2-4fdc-b364-14c63c900547","Type":"ContainerDied","Data":"e9c5d4664e6a6af1a280adc82f0f7baad6cecdbafdfe9445a91959cb79c54639"}
Jul 16 22:23:10.338414 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.338389    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7" event={"ID":"d560b140-abf1-42b7-bbe9-b907ed8d856e","Type":"ContainerDied","Data":"2dcd15f47e8562d57c375995503f3f9ac83e390eff0b32508360eb9d448566c6"}
Jul 16 22:23:10.338537 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.338435    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2dcd15f47e8562d57c375995503f3f9ac83e390eff0b32508360eb9d448566c6"
Jul 16 22:23:10.338537 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.338398    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7"
Jul 16 22:23:10.340263 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.340236    2564 generic.go:356] "Generic (PLEG): container finished" podID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerID="a1decc3afeabed8b1f1728a214c4d336872764f07f586a7658f34e7170072a5f" exitCode=0
Jul 16 22:23:10.340395 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.340329    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5" event={"ID":"d616be2e-4d83-4d91-8640-76078db8c7a4","Type":"ContainerDied","Data":"a1decc3afeabed8b1f1728a214c4d336872764f07f586a7658f34e7170072a5f"}
Jul 16 22:23:10.342545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.342521    2564 generic.go:356] "Generic (PLEG): container finished" podID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerID="d72b952da2c030bfdcac402c0f5110c9fb13db503b02b6d73add9642fce7dca3" exitCode=0
Jul 16 22:23:10.342637 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:10.342558    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj" event={"ID":"073f84a5-5a1d-40c1-987e-771ef6d09f3d","Type":"ContainerDied","Data":"d72b952da2c030bfdcac402c0f5110c9fb13db503b02b6d73add9642fce7dca3"}
Jul 16 22:23:11.493594 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.493566    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:11.519743 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.519716    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:11.541683 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.541659    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:11.566632 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.566605    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x74tw\" (UniqueName: \"kubernetes.io/projected/d616be2e-4d83-4d91-8640-76078db8c7a4-kube-api-access-x74tw\") pod \"d616be2e-4d83-4d91-8640-76078db8c7a4\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") "
Jul 16 22:23:11.566786 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.566671    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnmkt\" (UniqueName: \"kubernetes.io/projected/aef2396c-fea2-4fdc-b364-14c63c900547-kube-api-access-cnmkt\") pod \"aef2396c-fea2-4fdc-b364-14c63c900547\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") "
Jul 16 22:23:11.566786 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.566697    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-bundle\") pod \"d616be2e-4d83-4d91-8640-76078db8c7a4\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") "
Jul 16 22:23:11.566903 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.566880    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-util\") pod \"aef2396c-fea2-4fdc-b364-14c63c900547\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") "
Jul 16 22:23:11.566964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.566922    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-util\") pod \"d616be2e-4d83-4d91-8640-76078db8c7a4\" (UID: \"d616be2e-4d83-4d91-8640-76078db8c7a4\") "
Jul 16 22:23:11.567026 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.567012    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-bundle\") pod \"aef2396c-fea2-4fdc-b364-14c63c900547\" (UID: \"aef2396c-fea2-4fdc-b364-14c63c900547\") "
Jul 16 22:23:11.567682 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.567324    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-bundle" (OuterVolumeSpecName: "bundle") pod "d616be2e-4d83-4d91-8640-76078db8c7a4" (UID: "d616be2e-4d83-4d91-8640-76078db8c7a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:11.567908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.567868    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-bundle" (OuterVolumeSpecName: "bundle") pod "aef2396c-fea2-4fdc-b364-14c63c900547" (UID: "aef2396c-fea2-4fdc-b364-14c63c900547"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:11.569285 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.569257    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d616be2e-4d83-4d91-8640-76078db8c7a4-kube-api-access-x74tw" (OuterVolumeSpecName: "kube-api-access-x74tw") pod "d616be2e-4d83-4d91-8640-76078db8c7a4" (UID: "d616be2e-4d83-4d91-8640-76078db8c7a4"). InnerVolumeSpecName "kube-api-access-x74tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:23:11.569394 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.569313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef2396c-fea2-4fdc-b364-14c63c900547-kube-api-access-cnmkt" (OuterVolumeSpecName: "kube-api-access-cnmkt") pod "aef2396c-fea2-4fdc-b364-14c63c900547" (UID: "aef2396c-fea2-4fdc-b364-14c63c900547"). InnerVolumeSpecName "kube-api-access-cnmkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:23:11.573643 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.573615    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-util" (OuterVolumeSpecName: "util") pod "d616be2e-4d83-4d91-8640-76078db8c7a4" (UID: "d616be2e-4d83-4d91-8640-76078db8c7a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:11.573940 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.573917    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-util" (OuterVolumeSpecName: "util") pod "aef2396c-fea2-4fdc-b364-14c63c900547" (UID: "aef2396c-fea2-4fdc-b364-14c63c900547"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:11.668039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668000    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-util\") pod \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") "
Jul 16 22:23:11.668039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-bundle\") pod \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") "
Jul 16 22:23:11.668270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668085    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lnz5\" (UniqueName: \"kubernetes.io/projected/073f84a5-5a1d-40c1-987e-771ef6d09f3d-kube-api-access-2lnz5\") pod \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\" (UID: \"073f84a5-5a1d-40c1-987e-771ef6d09f3d\") "
Jul 16 22:23:11.668314 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668294    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.668314 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668306    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.668382 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668315    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x74tw\" (UniqueName: \"kubernetes.io/projected/d616be2e-4d83-4d91-8640-76078db8c7a4-kube-api-access-x74tw\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.668382 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668325    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnmkt\" (UniqueName: \"kubernetes.io/projected/aef2396c-fea2-4fdc-b364-14c63c900547-kube-api-access-cnmkt\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.668382 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668333    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d616be2e-4d83-4d91-8640-76078db8c7a4-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.668382 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668341    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aef2396c-fea2-4fdc-b364-14c63c900547-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.668585 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.668555    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-bundle" (OuterVolumeSpecName: "bundle") pod "073f84a5-5a1d-40c1-987e-771ef6d09f3d" (UID: "073f84a5-5a1d-40c1-987e-771ef6d09f3d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:11.670163 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.670125    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073f84a5-5a1d-40c1-987e-771ef6d09f3d-kube-api-access-2lnz5" (OuterVolumeSpecName: "kube-api-access-2lnz5") pod "073f84a5-5a1d-40c1-987e-771ef6d09f3d" (UID: "073f84a5-5a1d-40c1-987e-771ef6d09f3d"). InnerVolumeSpecName "kube-api-access-2lnz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:23:11.673312 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.673291    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-util" (OuterVolumeSpecName: "util") pod "073f84a5-5a1d-40c1-987e-771ef6d09f3d" (UID: "073f84a5-5a1d-40c1-987e-771ef6d09f3d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:23:11.769095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.769060    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-util\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.769095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.769091    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f84a5-5a1d-40c1-987e-771ef6d09f3d-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:11.769095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:11.769100    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lnz5\" (UniqueName: \"kubernetes.io/projected/073f84a5-5a1d-40c1-987e-771ef6d09f3d-kube-api-access-2lnz5\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:23:12.354209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.354177    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5"
Jul 16 22:23:12.354209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.354187    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5" event={"ID":"d616be2e-4d83-4d91-8640-76078db8c7a4","Type":"ContainerDied","Data":"a2288e546c0353db7785922efadc963cbe73c6b4049695354a55a9598b112470"}
Jul 16 22:23:12.354409 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.354238    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2288e546c0353db7785922efadc963cbe73c6b4049695354a55a9598b112470"
Jul 16 22:23:12.356340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.356238    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj"
Jul 16 22:23:12.356340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.356242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj" event={"ID":"073f84a5-5a1d-40c1-987e-771ef6d09f3d","Type":"ContainerDied","Data":"35b8b6bf1649949e1cf6af497969647cd3a763bf92ed6a05849698c42cf6e637"}
Jul 16 22:23:12.356340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.356285    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35b8b6bf1649949e1cf6af497969647cd3a763bf92ed6a05849698c42cf6e637"
Jul 16 22:23:12.358145 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.358112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9" event={"ID":"aef2396c-fea2-4fdc-b364-14c63c900547","Type":"ContainerDied","Data":"32705af6fd3927a48dbc1f310b0b513b81d648d7da754e7f76f116aa579602d9"}
Jul 16 22:23:12.358273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.358160    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32705af6fd3927a48dbc1f310b0b513b81d648d7da754e7f76f116aa579602d9"
Jul 16 22:23:12.358273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:12.358170    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9"
Jul 16 22:23:40.802357 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802276    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-864f587d6b-7bx2q"]
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802639    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802654    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802671    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802678    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802687    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802693    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802705    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802710    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802717    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802721    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802730    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802734    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802743    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802748    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802754    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802759    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="util"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802765    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802770    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802775    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802780    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802785    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802790    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="pull"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802797    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="extract"
Jul 16 22:23:40.802828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802802    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="extract"
Jul 16 22:23:40.803575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802847    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d616be2e-4d83-4d91-8640-76078db8c7a4" containerName="extract"
Jul 16 22:23:40.803575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802854    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="073f84a5-5a1d-40c1-987e-771ef6d09f3d" containerName="extract"
Jul 16 22:23:40.803575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802861    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d560b140-abf1-42b7-bbe9-b907ed8d856e" containerName="extract"
Jul 16 22:23:40.803575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.802866    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aef2396c-fea2-4fdc-b364-14c63c900547" containerName="extract"
Jul 16 22:23:40.807512 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.807489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.822776 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.822746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864f587d6b-7bx2q"]
Jul 16 22:23:40.928514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-service-ca\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.928687 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-oauth-config\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.928687 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928582    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-oauth-serving-cert\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.928687 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-config\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.928687 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928670    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-serving-cert\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.928687 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928686    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-trusted-ca-bundle\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:40.928846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:40.928776    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7b7l\" (UniqueName: \"kubernetes.io/projected/ab391318-7b5a-4bb0-8f28-6924abda9d89-kube-api-access-b7b7l\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.029948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.029908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7b7l\" (UniqueName: \"kubernetes.io/projected/ab391318-7b5a-4bb0-8f28-6924abda9d89-kube-api-access-b7b7l\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030112 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.029961    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-service-ca\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030112 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-oauth-config\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030112 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-oauth-serving-cert\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-config\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-serving-cert\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-trusted-ca-bundle\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030768 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-service-ca\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030879 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-oauth-serving-cert\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.030938 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.030917    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-config\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.031046 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.031026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab391318-7b5a-4bb0-8f28-6924abda9d89-trusted-ca-bundle\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.032675 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.032658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-serving-cert\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.032839 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.032821    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ab391318-7b5a-4bb0-8f28-6924abda9d89-console-oauth-config\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.039379 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.039354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7b7l\" (UniqueName: \"kubernetes.io/projected/ab391318-7b5a-4bb0-8f28-6924abda9d89-kube-api-access-b7b7l\") pod \"console-864f587d6b-7bx2q\" (UID: \"ab391318-7b5a-4bb0-8f28-6924abda9d89\") " pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.117200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.117107    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:41.253611 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.253584    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864f587d6b-7bx2q"]
Jul 16 22:23:41.255453 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:23:41.255419    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab391318_7b5a_4bb0_8f28_6924abda9d89.slice/crio-0f98bb91880cfb56979293117de139080ee48cde608239da1a2007987d841f42 WatchSource:0}: Error finding container 0f98bb91880cfb56979293117de139080ee48cde608239da1a2007987d841f42: Status 404 returned error can't find the container with id 0f98bb91880cfb56979293117de139080ee48cde608239da1a2007987d841f42
Jul 16 22:23:41.505554 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.505515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864f587d6b-7bx2q" event={"ID":"ab391318-7b5a-4bb0-8f28-6924abda9d89","Type":"ContainerStarted","Data":"45fe3a8721343a6c80e79a67412d31ec8be16bc6d1c3c1321bc0d5c23591c1f5"}
Jul 16 22:23:41.505737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.505577    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864f587d6b-7bx2q" event={"ID":"ab391318-7b5a-4bb0-8f28-6924abda9d89","Type":"ContainerStarted","Data":"0f98bb91880cfb56979293117de139080ee48cde608239da1a2007987d841f42"}
Jul 16 22:23:41.525972 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:41.525914    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-864f587d6b-7bx2q" podStartSLOduration=1.5258981249999999 podStartE2EDuration="1.525898125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:23:41.524539785 +0000 UTC m=+377.845095857" watchObservedRunningTime="2026-07-16 22:23:41.525898125 +0000 UTC m=+377.846454237"
Jul 16 22:23:50.533914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.533879    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"]
Jul 16 22:23:50.537274 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.537255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.540645 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.540623    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 22:23:50.540771 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.540666    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-shwv8\""
Jul 16 22:23:50.540915 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.540901    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 22:23:50.563494 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.563467    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"]
Jul 16 22:23:50.612673 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.612640    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/409146cb-e1d9-400d-adf9-1f6db69b4a6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9lq4t\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.612838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.612771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2cql\" (UniqueName: \"kubernetes.io/projected/409146cb-e1d9-400d-adf9-1f6db69b4a6c-kube-api-access-m2cql\") pod \"kuadrant-operator-controller-manager-55467d9f7-9lq4t\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.713806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.713768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/409146cb-e1d9-400d-adf9-1f6db69b4a6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9lq4t\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.714010 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.713846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2cql\" (UniqueName: \"kubernetes.io/projected/409146cb-e1d9-400d-adf9-1f6db69b4a6c-kube-api-access-m2cql\") pod \"kuadrant-operator-controller-manager-55467d9f7-9lq4t\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.714238 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.714213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/409146cb-e1d9-400d-adf9-1f6db69b4a6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9lq4t\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.734029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.733987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2cql\" (UniqueName: \"kubernetes.io/projected/409146cb-e1d9-400d-adf9-1f6db69b4a6c-kube-api-access-m2cql\") pod \"kuadrant-operator-controller-manager-55467d9f7-9lq4t\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.848554 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.848471    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:50.989157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:50.989114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"]
Jul 16 22:23:50.991082 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:23:50.991049    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409146cb_e1d9_400d_adf9_1f6db69b4a6c.slice/crio-cf6860e2bf11bee7012e66c34a0cd4ccbc98488609d28284bb1a111d50613dae WatchSource:0}: Error finding container cf6860e2bf11bee7012e66c34a0cd4ccbc98488609d28284bb1a111d50613dae: Status 404 returned error can't find the container with id cf6860e2bf11bee7012e66c34a0cd4ccbc98488609d28284bb1a111d50613dae
Jul 16 22:23:51.117360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:51.117268    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:51.117604 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:51.117569    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:51.122440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:51.122416    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:51.556952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:51.556907    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t" event={"ID":"409146cb-e1d9-400d-adf9-1f6db69b4a6c","Type":"ContainerStarted","Data":"cf6860e2bf11bee7012e66c34a0cd4ccbc98488609d28284bb1a111d50613dae"}
Jul 16 22:23:51.561789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:51.561762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-864f587d6b-7bx2q"
Jul 16 22:23:51.622308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:51.622273    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c87dbfd89-lcp6f"]
Jul 16 22:23:56.587988 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:56.587947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t" event={"ID":"409146cb-e1d9-400d-adf9-1f6db69b4a6c","Type":"ContainerStarted","Data":"b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244"}
Jul 16 22:23:56.588398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:56.588075    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:23:56.610816 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:23:56.610765    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t" podStartSLOduration=1.852683904 podStartE2EDuration="6.610744849s" totalImagesPullingTime="4.758060945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:23:56.608748507 +0000 UTC m=+392.929304580" watchObservedRunningTime="2026-07-16 22:23:56.610744849 +0000 UTC m=+392.931300922"
Jul 16 22:24:07.600990 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:07.600962    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:24:16.644317 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:16.644255    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c87dbfd89-lcp6f" podUID="ff717ef8-79aa-4827-b7c1-9506bf330e6d" containerName="console" containerID="cri-o://53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a" gracePeriod=15
Jul 16 22:24:16.892570 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:16.892527    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:24:17.057816 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.057778    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-config\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.057855    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-serving-cert\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.057879    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-oauth-serving-cert\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.057925    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-trusted-ca-bundle\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.057948    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-oauth-config\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.057995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkcvc\" (UniqueName: \"kubernetes.io/projected/ff717ef8-79aa-4827-b7c1-9506bf330e6d-kube-api-access-pkcvc\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.058027    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-service-ca\") pod \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\" (UID: \"ff717ef8-79aa-4827-b7c1-9506bf330e6d\") "
Jul 16 22:24:17.058702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.058496    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-config" (OuterVolumeSpecName: "console-config") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:24:17.058702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.058597    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:24:17.058702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.058653    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:24:17.058702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.058685    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:24:17.060264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.060241    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff717ef8-79aa-4827-b7c1-9506bf330e6d-kube-api-access-pkcvc" (OuterVolumeSpecName: "kube-api-access-pkcvc") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "kube-api-access-pkcvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:24:17.060264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.060248    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:24:17.060389 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.060269    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff717ef8-79aa-4827-b7c1-9506bf330e6d" (UID: "ff717ef8-79aa-4827-b7c1-9506bf330e6d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:24:17.159593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159548    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.159593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159581    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.159593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159590    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.159593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159601    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-oauth-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.159852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159612    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff717ef8-79aa-4827-b7c1-9506bf330e6d-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.159852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159625    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff717ef8-79aa-4827-b7c1-9506bf330e6d-console-oauth-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.159852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.159635    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkcvc\" (UniqueName: \"kubernetes.io/projected/ff717ef8-79aa-4827-b7c1-9506bf330e6d-kube-api-access-pkcvc\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:17.695348 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.695317    2564 generic.go:356] "Generic (PLEG): container finished" podID="ff717ef8-79aa-4827-b7c1-9506bf330e6d" containerID="53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a" exitCode=0
Jul 16 22:24:17.695745 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.695358    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87dbfd89-lcp6f"
Jul 16 22:24:17.695745 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.695391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87dbfd89-lcp6f" event={"ID":"ff717ef8-79aa-4827-b7c1-9506bf330e6d","Type":"ContainerDied","Data":"53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a"}
Jul 16 22:24:17.695745 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.695437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87dbfd89-lcp6f" event={"ID":"ff717ef8-79aa-4827-b7c1-9506bf330e6d","Type":"ContainerDied","Data":"f073c9b09707c90eab94da035f4cf20673130b0ea830d14f34106b85cec69b3b"}
Jul 16 22:24:17.695745 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.695458    2564 scope.go:117] "RemoveContainer" containerID="53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a"
Jul 16 22:24:17.704302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.704281    2564 scope.go:117] "RemoveContainer" containerID="53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a"
Jul 16 22:24:17.704597 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:24:17.704579    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a\": container with ID starting with 53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a not found: ID does not exist" containerID="53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a"
Jul 16 22:24:17.704668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.704606    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a"} err="failed to get container status \"53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a\": rpc error: code = NotFound desc = could not find container \"53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a\": container with ID starting with 53c4f419ca5ab079420b116d916aceadfe517bc25969ab298eca62ddae6ea94a not found: ID does not exist"
Jul 16 22:24:17.742582 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.742549    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c87dbfd89-lcp6f"]
Jul 16 22:24:17.763843 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:17.763813    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c87dbfd89-lcp6f"]
Jul 16 22:24:19.601904 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:19.601870    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff717ef8-79aa-4827-b7c1-9506bf330e6d" path="/var/lib/kubelet/pods/ff717ef8-79aa-4827-b7c1-9506bf330e6d/volumes"
Jul 16 22:24:20.032738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.032699    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"]
Jul 16 22:24:20.032947 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.032923    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t" podUID="409146cb-e1d9-400d-adf9-1f6db69b4a6c" containerName="manager" containerID="cri-o://b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244" gracePeriod=10
Jul 16 22:24:20.282670 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.282646    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:24:20.386708 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.386595    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2cql\" (UniqueName: \"kubernetes.io/projected/409146cb-e1d9-400d-adf9-1f6db69b4a6c-kube-api-access-m2cql\") pod \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") "
Jul 16 22:24:20.386708 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.386662    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/409146cb-e1d9-400d-adf9-1f6db69b4a6c-extensions-socket-volume\") pod \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\" (UID: \"409146cb-e1d9-400d-adf9-1f6db69b4a6c\") "
Jul 16 22:24:20.387253 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.387224    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409146cb-e1d9-400d-adf9-1f6db69b4a6c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "409146cb-e1d9-400d-adf9-1f6db69b4a6c" (UID: "409146cb-e1d9-400d-adf9-1f6db69b4a6c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:24:20.388726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.388704    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/409146cb-e1d9-400d-adf9-1f6db69b4a6c-kube-api-access-m2cql" (OuterVolumeSpecName: "kube-api-access-m2cql") pod "409146cb-e1d9-400d-adf9-1f6db69b4a6c" (UID: "409146cb-e1d9-400d-adf9-1f6db69b4a6c"). InnerVolumeSpecName "kube-api-access-m2cql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:24:20.487383 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.487328    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2cql\" (UniqueName: \"kubernetes.io/projected/409146cb-e1d9-400d-adf9-1f6db69b4a6c-kube-api-access-m2cql\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:20.487383 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.487374    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/409146cb-e1d9-400d-adf9-1f6db69b4a6c-extensions-socket-volume\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:24:20.717865 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.717833    2564 generic.go:356] "Generic (PLEG): container finished" podID="409146cb-e1d9-400d-adf9-1f6db69b4a6c" containerID="b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244" exitCode=0
Jul 16 22:24:20.718334 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.717877    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"
Jul 16 22:24:20.718334 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.717911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t" event={"ID":"409146cb-e1d9-400d-adf9-1f6db69b4a6c","Type":"ContainerDied","Data":"b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244"}
Jul 16 22:24:20.718334 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.717956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t" event={"ID":"409146cb-e1d9-400d-adf9-1f6db69b4a6c","Type":"ContainerDied","Data":"cf6860e2bf11bee7012e66c34a0cd4ccbc98488609d28284bb1a111d50613dae"}
Jul 16 22:24:20.718334 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.717979    2564 scope.go:117] "RemoveContainer" containerID="b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244"
Jul 16 22:24:20.727376 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.727354    2564 scope.go:117] "RemoveContainer" containerID="b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244"
Jul 16 22:24:20.727667 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:24:20.727646    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244\": container with ID starting with b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244 not found: ID does not exist" containerID="b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244"
Jul 16 22:24:20.727714 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.727675    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244"} err="failed to get container status \"b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244\": rpc error: code = NotFound desc = could not find container \"b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244\": container with ID starting with b93ec4bfb9027e5855ce1624f31af8fba343f96fbab649bc3f780d675f91c244 not found: ID does not exist"
Jul 16 22:24:20.742997 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.742960    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"]
Jul 16 22:24:20.747506 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:20.747479    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9lq4t"]
Jul 16 22:24:21.601195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:21.601158    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="409146cb-e1d9-400d-adf9-1f6db69b4a6c" path="/var/lib/kubelet/pods/409146cb-e1d9-400d-adf9-1f6db69b4a6c/volumes"
Jul 16 22:24:24.023619 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023583    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"]
Jul 16 22:24:24.023978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023887    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff717ef8-79aa-4827-b7c1-9506bf330e6d" containerName="console"
Jul 16 22:24:24.023978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023897    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff717ef8-79aa-4827-b7c1-9506bf330e6d" containerName="console"
Jul 16 22:24:24.023978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023916    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409146cb-e1d9-400d-adf9-1f6db69b4a6c" containerName="manager"
Jul 16 22:24:24.023978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023921    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="409146cb-e1d9-400d-adf9-1f6db69b4a6c" containerName="manager"
Jul 16 22:24:24.023978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023968    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="409146cb-e1d9-400d-adf9-1f6db69b4a6c" containerName="manager"
Jul 16 22:24:24.023978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.023979    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff717ef8-79aa-4827-b7c1-9506bf330e6d" containerName="console"
Jul 16 22:24:24.028097 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.028080    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.031743 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.031726    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2qq7c\""
Jul 16 22:24:24.059195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.059168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"]
Jul 16 22:24:24.120726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.120726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.120987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120742    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjkd9\" (UniqueName: \"kubernetes.io/projected/328a244c-177f-4f43-9187-8338e3e80b73-kube-api-access-jjkd9\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.120987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.120987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328a244c-177f-4f43-9187-8338e3e80b73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.120987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328a244c-177f-4f43-9187-8338e3e80b73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.120987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.120978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.121207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.121018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328a244c-177f-4f43-9187-8338e3e80b73-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.121207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.121050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.221804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.221765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.221804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.221807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328a244c-177f-4f43-9187-8338e3e80b73-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.221832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.221953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.221984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222003    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjkd9\" (UniqueName: \"kubernetes.io/projected/328a244c-177f-4f43-9187-8338e3e80b73-kube-api-access-jjkd9\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328a244c-177f-4f43-9187-8338e3e80b73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328a244c-177f-4f43-9187-8338e3e80b73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222516 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222516 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222424    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.222897 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.222877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328a244c-177f-4f43-9187-8338e3e80b73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.224240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.224212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328a244c-177f-4f43-9187-8338e3e80b73-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.224512 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.224495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328a244c-177f-4f43-9187-8338e3e80b73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.230376 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.230353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328a244c-177f-4f43-9187-8338e3e80b73-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.231083 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.231055    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjkd9\" (UniqueName: \"kubernetes.io/projected/328a244c-177f-4f43-9187-8338e3e80b73-kube-api-access-jjkd9\") pod \"maas-default-gateway-openshift-default-56654dcf7-mhh8v\" (UID: \"328a244c-177f-4f43-9187-8338e3e80b73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.341084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.341057    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2qq7c\""
Jul 16 22:24:24.349107 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.349084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:24.483822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.483796    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"]
Jul 16 22:24:24.485577 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:24:24.485546    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328a244c_177f_4f43_9187_8338e3e80b73.slice/crio-d7dab05e5fc8e0045cf982818f3ad09927873bc768732a8d246daf8c37fc6eca WatchSource:0}: Error finding container d7dab05e5fc8e0045cf982818f3ad09927873bc768732a8d246daf8c37fc6eca: Status 404 returned error can't find the container with id d7dab05e5fc8e0045cf982818f3ad09927873bc768732a8d246daf8c37fc6eca
Jul 16 22:24:24.487769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.487729    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:24:24.487841 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.487811    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:24:24.488056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.487853    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:24:24.740039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.740007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" event={"ID":"328a244c-177f-4f43-9187-8338e3e80b73","Type":"ContainerStarted","Data":"b29f73abbadb10a5516bbb60aca0f6f33e9682c4f27b81c3247dcd5ff85694c0"}
Jul 16 22:24:24.740200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.740056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" event={"ID":"328a244c-177f-4f43-9187-8338e3e80b73","Type":"ContainerStarted","Data":"d7dab05e5fc8e0045cf982818f3ad09927873bc768732a8d246daf8c37fc6eca"}
Jul 16 22:24:24.766405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:24.766347    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" podStartSLOduration=1.766327035 podStartE2EDuration="1.766327035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:24.764953034 +0000 UTC m=+421.085509106" watchObservedRunningTime="2026-07-16 22:24:24.766327035 +0000 UTC m=+421.086883108"
Jul 16 22:24:25.349450 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:25.349410    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:25.354168 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:25.354121    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:25.745195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:25.745159    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:25.746061 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:25.746039    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:24:38.556331 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.556293    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:24:38.559672 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.559647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.564614 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.564590    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 22:24:38.564852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.564838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r9mf7\""
Jul 16 22:24:38.564852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.564843    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 22:24:38.565520 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.565501    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 22:24:38.570102 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.570081    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:24:38.601953 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.601924    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:24:38.657400 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.657364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c4d21-d45a-4c78-a2d2-8b056e30a7ed-config-file\") pod \"limitador-limitador-54d6d988d5-sdqkx\" (UID: \"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.657653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.657551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbhgb\" (UniqueName: \"kubernetes.io/projected/a10c4d21-d45a-4c78-a2d2-8b056e30a7ed-kube-api-access-gbhgb\") pod \"limitador-limitador-54d6d988d5-sdqkx\" (UID: \"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.759150 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.759105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c4d21-d45a-4c78-a2d2-8b056e30a7ed-config-file\") pod \"limitador-limitador-54d6d988d5-sdqkx\" (UID: \"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.759316 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.759255    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbhgb\" (UniqueName: \"kubernetes.io/projected/a10c4d21-d45a-4c78-a2d2-8b056e30a7ed-kube-api-access-gbhgb\") pod \"limitador-limitador-54d6d988d5-sdqkx\" (UID: \"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.759746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.759724    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a10c4d21-d45a-4c78-a2d2-8b056e30a7ed-config-file\") pod \"limitador-limitador-54d6d988d5-sdqkx\" (UID: \"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.769581 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.769545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbhgb\" (UniqueName: \"kubernetes.io/projected/a10c4d21-d45a-4c78-a2d2-8b056e30a7ed-kube-api-access-gbhgb\") pod \"limitador-limitador-54d6d988d5-sdqkx\" (UID: \"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:38.871100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:38.870998    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:39.004384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:39.004357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:24:39.005934 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:24:39.005903    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10c4d21_d45a_4c78_a2d2_8b056e30a7ed.slice/crio-58224b7b89eb63652f23d07e2e70a9954de4e5e1551dcb957e92ef5bbecc17ac WatchSource:0}: Error finding container 58224b7b89eb63652f23d07e2e70a9954de4e5e1551dcb957e92ef5bbecc17ac: Status 404 returned error can't find the container with id 58224b7b89eb63652f23d07e2e70a9954de4e5e1551dcb957e92ef5bbecc17ac
Jul 16 22:24:39.814385 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:39.814347    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx" event={"ID":"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed","Type":"ContainerStarted","Data":"58224b7b89eb63652f23d07e2e70a9954de4e5e1551dcb957e92ef5bbecc17ac"}
Jul 16 22:24:42.831297 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:42.831258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx" event={"ID":"a10c4d21-d45a-4c78-a2d2-8b056e30a7ed","Type":"ContainerStarted","Data":"21c47eba712710667f6b05ae1cfb2bef36b6c9edc3cb14c7980b82dfe85afd29"}
Jul 16 22:24:42.831680 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:42.831402    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:24:42.850849 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:42.850792    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx" podStartSLOduration=1.752394979 podStartE2EDuration="4.850774858s" totalImagesPullingTime="3.098379879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:42.848506884 +0000 UTC m=+439.169062957" watchObservedRunningTime="2026-07-16 22:24:42.850774858 +0000 UTC m=+439.171330952"
Jul 16 22:24:53.837538 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:24:53.837451    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"
Jul 16 22:25:53.695732 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:25:53.695695    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:26:01.222656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:01.222571    2564 generic.go:356] "Generic (PLEG): container finished" podID="328a244c-177f-4f43-9187-8338e3e80b73" containerID="b29f73abbadb10a5516bbb60aca0f6f33e9682c4f27b81c3247dcd5ff85694c0" exitCode=0
Jul 16 22:26:01.222656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:01.222616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" event={"ID":"328a244c-177f-4f43-9187-8338e3e80b73","Type":"ContainerDied","Data":"b29f73abbadb10a5516bbb60aca0f6f33e9682c4f27b81c3247dcd5ff85694c0"}
Jul 16 22:26:01.223067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:01.223003    2564 scope.go:117] "RemoveContainer" containerID="b29f73abbadb10a5516bbb60aca0f6f33e9682c4f27b81c3247dcd5ff85694c0"
Jul 16 22:26:01.224028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:01.224001    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:26:01.224092 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:01.224069    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:26:01.224127 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:01.224101    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:26:02.229905 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:02.229871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" event={"ID":"328a244c-177f-4f43-9187-8338e3e80b73","Type":"ContainerStarted","Data":"59113f3f03496120ef23ff486b6dbe05d2aa9268f3f3c5b0d4c28e19388d9486"}
Jul 16 22:26:02.349767 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:02.349728    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:26:02.349962 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:02.349940    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mhh8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 16 22:26:02.350017 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:02.349992    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" podUID="328a244c-177f-4f43-9187-8338e3e80b73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 16 22:26:03.350293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:03.350248    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mhh8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 16 22:26:03.350670 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:03.350304    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" podUID="328a244c-177f-4f43-9187-8338e3e80b73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 16 22:26:04.350050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:04.350013    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mhh8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 16 22:26:04.350248 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:04.350069    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" podUID="328a244c-177f-4f43-9187-8338e3e80b73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 16 22:26:05.350524 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:05.350480    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mhh8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 16 22:26:05.350915 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:05.350539    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v" podUID="328a244c-177f-4f43-9187-8338e3e80b73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 16 22:26:06.353158 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:06.353113    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:26:06.353576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:06.353373    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:26:06.354247 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:06.354227    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mhh8v"
Jul 16 22:26:10.578251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:10.578215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:26:26.278324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:26.278242    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:26:32.223123 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:32.223086    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:26:39.977273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:39.977230    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:26:49.284538 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:26:49.284497    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:27:22.086360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:27:22.086325    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:27:24.337311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:27:24.337282    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:27:24.337738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:27:24.337286    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:29:10.489846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:29:10.489810    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:29:23.377359 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:29:23.377275    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:29:32.277170 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:29:32.277118    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:30:00.144125 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.144086    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"]
Jul 16 22:30:00.146215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.146197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"
Jul 16 22:30:00.148519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.148496    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:30:00.154671 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.154635    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"]
Jul 16 22:30:00.156947 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.156925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"
Jul 16 22:30:00.286929 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.286904    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"]
Jul 16 22:30:00.288544 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:30:00.288515    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f6ae87_5c45_4260_8bb7_031714c32321.slice/crio-3c039513f925d4245bdf26acc4bc1989dc5976a706dfc08365342758500ff59f WatchSource:0}: Error finding container 3c039513f925d4245bdf26acc4bc1989dc5976a706dfc08365342758500ff59f: Status 404 returned error can't find the container with id 3c039513f925d4245bdf26acc4bc1989dc5976a706dfc08365342758500ff59f
Jul 16 22:30:00.290446 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.290429    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:30:00.392501 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:00.392465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq" event={"ID":"b9f6ae87-5c45-4260-8bb7-031714c32321","Type":"ContainerStarted","Data":"3c039513f925d4245bdf26acc4bc1989dc5976a706dfc08365342758500ff59f"}
Jul 16 22:30:03.412649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:03.412613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq" event={"ID":"b9f6ae87-5c45-4260-8bb7-031714c32321","Type":"ContainerStarted","Data":"6a6fcbf747be78fee1e33cee4b745f79ce73126d9f5f395193cb0605ecde4672"}
Jul 16 22:30:03.428934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:03.428881    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq" podStartSLOduration=1.090322853 podStartE2EDuration="3.42886512s" totalImagesPullingTime="2.338542267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:30:03.426677703 +0000 UTC m=+759.747233757" watchObservedRunningTime="2026-07-16 22:30:03.42886512 +0000 UTC m=+759.749421191"
Jul 16 22:30:04.419306 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:04.419271    2564 generic.go:356] "Generic (PLEG): container finished" podID="b9f6ae87-5c45-4260-8bb7-031714c32321" containerID="6a6fcbf747be78fee1e33cee4b745f79ce73126d9f5f395193cb0605ecde4672" exitCode=0
Jul 16 22:30:04.419678 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:04.419366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq" event={"ID":"b9f6ae87-5c45-4260-8bb7-031714c32321","Type":"ContainerDied","Data":"6a6fcbf747be78fee1e33cee4b745f79ce73126d9f5f395193cb0605ecde4672"}
Jul 16 22:30:05.554627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:05.554602    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"
Jul 16 22:30:06.431277 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:06.431248    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq"
Jul 16 22:30:06.431449 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:06.431240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-v7dxq" event={"ID":"b9f6ae87-5c45-4260-8bb7-031714c32321","Type":"ContainerDied","Data":"3c039513f925d4245bdf26acc4bc1989dc5976a706dfc08365342758500ff59f"}
Jul 16 22:30:06.431449 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:30:06.431393    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c039513f925d4245bdf26acc4bc1989dc5976a706dfc08365342758500ff59f"
Jul 16 22:31:11.325496 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:31:11.325458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:31:21.693147 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:31:21.693087    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:31:56.393816 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:31:56.393777    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:32:06.282103 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:06.282056    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:32:15.095077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:15.095046    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:32:24.369788 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:24.369697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:32:24.372193 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:24.372168    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:32:26.684150 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:26.684107    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:32:37.697730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:37.697696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:32:49.290804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:32:49.290770    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:33:00.405751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:33:00.405713    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:33:10.088960 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:33:10.088927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:33:32.696064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:33:32.696031    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:33:43.601180 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:33:43.601120    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:34:08.492424 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:34:08.492387    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:34:24.796202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:34:24.796161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:34:35.088907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:34:35.088866    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:34:51.218114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:34:51.218075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:35:05.390180 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:35:05.390129    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:35:14.197414 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:35:14.197370    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:37:02.504329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:02.504295    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:37:14.186298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:14.186261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:37:24.402527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:24.402498    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:37:24.407264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:24.407242    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:37:24.598529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:24.598494    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:37:34.999066 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:34.999027    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:37:45.610377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:45.610337    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:37:56.686794 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:37:56.686743    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:38:05.682298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:38:05.682265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:38:16.793531 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:38:16.793449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:38:27.692549 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:38:27.692509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:38:36.291930 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:38:36.291889    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:38:46.890206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:38:46.890171    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:39:08.599445 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:39:08.599409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:39:20.291849 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:39:20.291809    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:39:30.287363 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:39:30.287317    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:39:40.086186 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:39:40.086126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:39:48.495625 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:39:48.495539    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:40:01.681000 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:40:01.680965    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:40:15.296659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:40:15.296621    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:40:24.480735 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:40:24.480699    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:40:39.281171 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:40:39.281122    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:40:48.397013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:40:48.396970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:41:01.697266 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:41:01.697228    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:41:11.196340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:41:11.196301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:41:20.684206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:41:20.684110    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:41:31.591547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:41:31.591509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:41:44.603122 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:41:44.603087    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:41:53.896926 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:41:53.896888    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:42:06.492802 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:06.492759    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:42:18.530573 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:18.530525    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:42:24.434656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:24.434627    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:42:24.444272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:24.444252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:42:31.797647 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:31.797612    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:42:45.303079 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:45.303044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:42:58.060549 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:42:58.060516    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:43:05.109813 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.109773    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"]
Jul 16 22:43:05.110254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.110149    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f6ae87-5c45-4260-8bb7-031714c32321" containerName="cleanup"
Jul 16 22:43:05.110254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.110160    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f6ae87-5c45-4260-8bb7-031714c32321" containerName="cleanup"
Jul 16 22:43:05.110254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.110226    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9f6ae87-5c45-4260-8bb7-031714c32321" containerName="cleanup"
Jul 16 22:43:05.113163 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.113127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.115480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.115447    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5cb4a56a-openshift-default-dockercfg-zgwng\""
Jul 16 22:43:05.211893 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.211856    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"]
Jul 16 22:43:05.230668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-data\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.230668 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-podinfo\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.230886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knxmv\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-kube-api-access-knxmv\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.230886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230754    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-envoy\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.230886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230825    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-credential-socket\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.230886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-token\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.230886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230854    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-socket\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.231343 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-certs\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.231343 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.230989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istiod-ca-cert\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.331859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.331813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-credential-socket\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.331859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.331861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-token\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.331966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-socket\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.331993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-certs\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istiod-ca-cert\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-data\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-podinfo\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332413 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knxmv\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-kube-api-access-knxmv\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332413 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-envoy\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332413 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332233    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-credential-socket\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332413 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-socket\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332690 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-data\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.332736 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.332680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-certs\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.333065 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.333043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istiod-ca-cert\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.334481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.334458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-envoy\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.334704 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.334685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-podinfo\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.342201 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.342147    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-token\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.344709 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.344690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knxmv\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-kube-api-access-knxmv\") pod \"e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") " pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.423820 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.423780    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:05.578553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.578517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"]
Jul 16 22:43:05.579910 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:43:05.579886    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e292c2_e67b_4d0b_ae99_365d2d1da5f8.slice/crio-6ce18b8de8693de2af505d9a85e048f83c48407f5d387702249aed71372ddae8 WatchSource:0}: Error finding container 6ce18b8de8693de2af505d9a85e048f83c48407f5d387702249aed71372ddae8: Status 404 returned error can't find the container with id 6ce18b8de8693de2af505d9a85e048f83c48407f5d387702249aed71372ddae8
Jul 16 22:43:05.582067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.582037    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:43:05.582116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.582099    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:43:05.582165 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:05.582127    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:43:06.334611 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:06.334575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" event={"ID":"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8","Type":"ContainerStarted","Data":"41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"}
Jul 16 22:43:06.335010 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:06.334629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" event={"ID":"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8","Type":"ContainerStarted","Data":"6ce18b8de8693de2af505d9a85e048f83c48407f5d387702249aed71372ddae8"}
Jul 16 22:43:06.392268 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:06.392213    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" podStartSLOduration=1.392195114 podStartE2EDuration="1.392195114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:43:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:43:06.39033501 +0000 UTC m=+1542.710891082" watchObservedRunningTime="2026-07-16 22:43:06.392195114 +0000 UTC m=+1542.712751188"
Jul 16 22:43:06.424234 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:06.424200    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:06.429115 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:06.429085    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:07.340095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:07.340068    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:07.340977 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:07.340961    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:10.890035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:10.890001    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:43:11.289008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.288967    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"]
Jul 16 22:43:11.292501 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.292483    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:11.294968 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.294943    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:43:11.295090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.295005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:43:11.301776 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.301752    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"]
Jul 16 22:43:11.388989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.388950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134acb00-0f05-4e19-bbd5-f8801b8052f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7\" (UID: \"134acb00-0f05-4e19-bbd5-f8801b8052f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:11.490329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.490289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134acb00-0f05-4e19-bbd5-f8801b8052f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7\" (UID: \"134acb00-0f05-4e19-bbd5-f8801b8052f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:11.490924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.490904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134acb00-0f05-4e19-bbd5-f8801b8052f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7\" (UID: \"134acb00-0f05-4e19-bbd5-f8801b8052f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:11.605101 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.605025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:11.953900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:11.953878    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"]
Jul 16 22:43:11.955753 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:43:11.955724    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134acb00_0f05_4e19_bbd5_f8801b8052f8.slice/crio-0a8ed3d93f9d687a0c45906d2889e4613cccde6fbdb637189270ad5148aff84d WatchSource:0}: Error finding container 0a8ed3d93f9d687a0c45906d2889e4613cccde6fbdb637189270ad5148aff84d: Status 404 returned error can't find the container with id 0a8ed3d93f9d687a0c45906d2889e4613cccde6fbdb637189270ad5148aff84d
Jul 16 22:43:12.366500 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:12.366463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" event={"ID":"134acb00-0f05-4e19-bbd5-f8801b8052f8","Type":"ContainerStarted","Data":"7fe62f7f2bcb64694faad10161cdeee9b078af174a6b8cbbb8ba61fe92ddbdcc"}
Jul 16 22:43:12.366699 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:12.366523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" event={"ID":"134acb00-0f05-4e19-bbd5-f8801b8052f8","Type":"ContainerStarted","Data":"0a8ed3d93f9d687a0c45906d2889e4613cccde6fbdb637189270ad5148aff84d"}
Jul 16 22:43:12.386013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:12.385938    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" podStartSLOduration=1.385901726 podStartE2EDuration="1.385901726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:43:12.383554419 +0000 UTC m=+1548.704110494" watchObservedRunningTime="2026-07-16 22:43:12.385901726 +0000 UTC m=+1548.706457800"
Jul 16 22:43:13.376250 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:13.376150    2564 generic.go:356] "Generic (PLEG): container finished" podID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerID="7fe62f7f2bcb64694faad10161cdeee9b078af174a6b8cbbb8ba61fe92ddbdcc" exitCode=7
Jul 16 22:43:13.376250 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:13.376233    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" event={"ID":"134acb00-0f05-4e19-bbd5-f8801b8052f8","Type":"ContainerDied","Data":"7fe62f7f2bcb64694faad10161cdeee9b078af174a6b8cbbb8ba61fe92ddbdcc"}
Jul 16 22:43:13.376777 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:13.376592    2564 scope.go:117] "RemoveContainer" containerID="7fe62f7f2bcb64694faad10161cdeee9b078af174a6b8cbbb8ba61fe92ddbdcc"
Jul 16 22:43:14.403221 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:14.403186    2564 generic.go:356] "Generic (PLEG): container finished" podID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerID="3fa4a1e0e4ad041fcb038b7b62b2fc940aadadd15bc671fed199fd4a603e3cf6" exitCode=7
Jul 16 22:43:14.403648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:14.403249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" event={"ID":"134acb00-0f05-4e19-bbd5-f8801b8052f8","Type":"ContainerDied","Data":"3fa4a1e0e4ad041fcb038b7b62b2fc940aadadd15bc671fed199fd4a603e3cf6"}
Jul 16 22:43:14.403648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:14.403320    2564 scope.go:117] "RemoveContainer" containerID="7fe62f7f2bcb64694faad10161cdeee9b078af174a6b8cbbb8ba61fe92ddbdcc"
Jul 16 22:43:14.403648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:14.403496    2564 scope.go:117] "RemoveContainer" containerID="3fa4a1e0e4ad041fcb038b7b62b2fc940aadadd15bc671fed199fd4a603e3cf6"
Jul 16 22:43:14.403771 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:43:14.403751    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7_odh-ai-gateway-infra(134acb00-0f05-4e19-bbd5-f8801b8052f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8"
Jul 16 22:43:15.410547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:15.410459    2564 scope.go:117] "RemoveContainer" containerID="3fa4a1e0e4ad041fcb038b7b62b2fc940aadadd15bc671fed199fd4a603e3cf6"
Jul 16 22:43:15.410900 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:43:15.410673    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7_odh-ai-gateway-infra(134acb00-0f05-4e19-bbd5-f8801b8052f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8"
Jul 16 22:43:19.430925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:19.430889    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerID="41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2" exitCode=0
Jul 16 22:43:19.431491 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:19.430965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" event={"ID":"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8","Type":"ContainerDied","Data":"41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"}
Jul 16 22:43:19.431567 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:19.431492    2564 scope.go:117] "RemoveContainer" containerID="41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"
Jul 16 22:43:19.432503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:19.432474    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:43:19.432620 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:19.432536    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:43:19.432620 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:19.432588    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:43:20.438170 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:20.438118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" event={"ID":"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8","Type":"ContainerStarted","Data":"68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d"}
Jul 16 22:43:21.424566 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:21.424525    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:21.425558 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:21.425533    2564 patch_prober.go:28] interesting pod/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 16 22:43:21.425624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:21.425585    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 16 22:43:22.340357 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:22.340322    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:22.424727 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:22.424688    2564 patch_prober.go:28] interesting pod/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 16 22:43:22.424905 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:22.424758    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 16 22:43:23.424685 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:23.424649    2564 patch_prober.go:28] interesting pod/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 16 22:43:23.425077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:23.424711    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 16 22:43:24.424580 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:24.424541    2564 patch_prober.go:28] interesting pod/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 16 22:43:24.424770 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:24.424606    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 16 22:43:25.428664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:25.428625    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:25.429579 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:25.429562    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:43:28.597036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:28.597005    2564 scope.go:117] "RemoveContainer" containerID="3fa4a1e0e4ad041fcb038b7b62b2fc940aadadd15bc671fed199fd4a603e3cf6"
Jul 16 22:43:29.485147 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:29.485105    2564 generic.go:356] "Generic (PLEG): container finished" podID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerID="ffe91515095455181faa34b06b967ddf12cf74e8504143592e23aad9a5c39acf" exitCode=7
Jul 16 22:43:29.485320 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:29.485168    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" event={"ID":"134acb00-0f05-4e19-bbd5-f8801b8052f8","Type":"ContainerDied","Data":"ffe91515095455181faa34b06b967ddf12cf74e8504143592e23aad9a5c39acf"}
Jul 16 22:43:29.485320 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:29.485227    2564 scope.go:117] "RemoveContainer" containerID="3fa4a1e0e4ad041fcb038b7b62b2fc940aadadd15bc671fed199fd4a603e3cf6"
Jul 16 22:43:29.485513 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:29.485496    2564 scope.go:117] "RemoveContainer" containerID="ffe91515095455181faa34b06b967ddf12cf74e8504143592e23aad9a5c39acf"
Jul 16 22:43:29.485705 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:43:29.485685    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7_odh-ai-gateway-infra(134acb00-0f05-4e19-bbd5-f8801b8052f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8"
Jul 16 22:43:30.535942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:30.535905    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"]
Jul 16 22:43:30.673380 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:30.673355    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:30.766949 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:30.766912    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134acb00-0f05-4e19-bbd5-f8801b8052f8-maas-api-service-ca\") pod \"134acb00-0f05-4e19-bbd5-f8801b8052f8\" (UID: \"134acb00-0f05-4e19-bbd5-f8801b8052f8\") "
Jul 16 22:43:30.767310 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:30.767288    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134acb00-0f05-4e19-bbd5-f8801b8052f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "134acb00-0f05-4e19-bbd5-f8801b8052f8" (UID: "134acb00-0f05-4e19-bbd5-f8801b8052f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:43:30.868323 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:30.868229    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134acb00-0f05-4e19-bbd5-f8801b8052f8-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:43:31.498190 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:31.498159    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"
Jul 16 22:43:31.498368 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:31.498192    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7" event={"ID":"134acb00-0f05-4e19-bbd5-f8801b8052f8","Type":"ContainerDied","Data":"0a8ed3d93f9d687a0c45906d2889e4613cccde6fbdb637189270ad5148aff84d"}
Jul 16 22:43:31.498368 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:31.498255    2564 scope.go:117] "RemoveContainer" containerID="ffe91515095455181faa34b06b967ddf12cf74e8504143592e23aad9a5c39acf"
Jul 16 22:43:31.521442 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:31.521411    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"]
Jul 16 22:43:31.524952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:31.524923    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lqjk7"]
Jul 16 22:43:31.601500 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:43:31.601471    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" path="/var/lib/kubelet/pods/134acb00-0f05-4e19-bbd5-f8801b8052f8/volumes"
Jul 16 22:44:11.347565 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347527    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"]
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347848    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347858    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347873    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347879    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347953    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347963    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.348054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.347968    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:44:11.350867 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.350852    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:11.353848 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.353829    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:44:11.354631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.354614    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:44:11.362339 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.362315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"]
Jul 16 22:44:11.429778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.429733    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b35a63-1546-4fe4-8271-c220a83a5267-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2\" (UID: \"24b35a63-1546-4fe4-8271-c220a83a5267\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:11.531063 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.531027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b35a63-1546-4fe4-8271-c220a83a5267-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2\" (UID: \"24b35a63-1546-4fe4-8271-c220a83a5267\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:11.531663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.531644    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b35a63-1546-4fe4-8271-c220a83a5267-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2\" (UID: \"24b35a63-1546-4fe4-8271-c220a83a5267\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:11.661339 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.661253    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:11.793787 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:11.793757    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"]
Jul 16 22:44:11.795635 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:44:11.795609    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b35a63_1546_4fe4_8271_c220a83a5267.slice/crio-94054c9ad2023c540b864cda2eddf19e97c092562e1c1a8b0c97d8ca4e821efb WatchSource:0}: Error finding container 94054c9ad2023c540b864cda2eddf19e97c092562e1c1a8b0c97d8ca4e821efb: Status 404 returned error can't find the container with id 94054c9ad2023c540b864cda2eddf19e97c092562e1c1a8b0c97d8ca4e821efb
Jul 16 22:44:12.709109 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:12.709073    2564 generic.go:356] "Generic (PLEG): container finished" podID="24b35a63-1546-4fe4-8271-c220a83a5267" containerID="85f8c8ab96bbdacd3dbd3b2a027e28fb8282e8da664ba62adc5c9f51908f1092" exitCode=7
Jul 16 22:44:12.709595 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:12.709157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" event={"ID":"24b35a63-1546-4fe4-8271-c220a83a5267","Type":"ContainerDied","Data":"85f8c8ab96bbdacd3dbd3b2a027e28fb8282e8da664ba62adc5c9f51908f1092"}
Jul 16 22:44:12.709595 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:12.709254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" event={"ID":"24b35a63-1546-4fe4-8271-c220a83a5267","Type":"ContainerStarted","Data":"94054c9ad2023c540b864cda2eddf19e97c092562e1c1a8b0c97d8ca4e821efb"}
Jul 16 22:44:12.709595 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:12.709334    2564 scope.go:117] "RemoveContainer" containerID="85f8c8ab96bbdacd3dbd3b2a027e28fb8282e8da664ba62adc5c9f51908f1092"
Jul 16 22:44:13.715944 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:13.715910    2564 generic.go:356] "Generic (PLEG): container finished" podID="24b35a63-1546-4fe4-8271-c220a83a5267" containerID="66e29a34955f42464b68fcf196b58430a554e88bea0881cbaaafdb3f23fd476d" exitCode=7
Jul 16 22:44:13.716441 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:13.715970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" event={"ID":"24b35a63-1546-4fe4-8271-c220a83a5267","Type":"ContainerDied","Data":"66e29a34955f42464b68fcf196b58430a554e88bea0881cbaaafdb3f23fd476d"}
Jul 16 22:44:13.716441 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:13.716025    2564 scope.go:117] "RemoveContainer" containerID="85f8c8ab96bbdacd3dbd3b2a027e28fb8282e8da664ba62adc5c9f51908f1092"
Jul 16 22:44:13.716441 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:13.716262    2564 scope.go:117] "RemoveContainer" containerID="66e29a34955f42464b68fcf196b58430a554e88bea0881cbaaafdb3f23fd476d"
Jul 16 22:44:13.716603 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:44:13.716509    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2_odh-ai-gateway-infra(24b35a63-1546-4fe4-8271-c220a83a5267)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" podUID="24b35a63-1546-4fe4-8271-c220a83a5267"
Jul 16 22:44:14.722930 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:14.722902    2564 scope.go:117] "RemoveContainer" containerID="66e29a34955f42464b68fcf196b58430a554e88bea0881cbaaafdb3f23fd476d"
Jul 16 22:44:14.723312 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:44:14.723083    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2_odh-ai-gateway-infra(24b35a63-1546-4fe4-8271-c220a83a5267)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" podUID="24b35a63-1546-4fe4-8271-c220a83a5267"
Jul 16 22:44:28.595977 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:28.595941    2564 scope.go:117] "RemoveContainer" containerID="66e29a34955f42464b68fcf196b58430a554e88bea0881cbaaafdb3f23fd476d"
Jul 16 22:44:28.795823 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:28.795797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" event={"ID":"24b35a63-1546-4fe4-8271-c220a83a5267","Type":"ContainerStarted","Data":"e04f47da6c0a3a36daf84394784de4cab97eba9fd9c0d5c7d75a860c20da0467"}
Jul 16 22:44:28.814453 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:28.814400    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" podStartSLOduration=17.814384787 podStartE2EDuration="17.814384787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:44:28.813584329 +0000 UTC m=+1625.134140400" watchObservedRunningTime="2026-07-16 22:44:28.814384787 +0000 UTC m=+1625.134940858"
Jul 16 22:44:29.802417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:29.802382    2564 generic.go:356] "Generic (PLEG): container finished" podID="24b35a63-1546-4fe4-8271-c220a83a5267" containerID="e04f47da6c0a3a36daf84394784de4cab97eba9fd9c0d5c7d75a860c20da0467" exitCode=7
Jul 16 22:44:29.802908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:29.802457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" event={"ID":"24b35a63-1546-4fe4-8271-c220a83a5267","Type":"ContainerDied","Data":"e04f47da6c0a3a36daf84394784de4cab97eba9fd9c0d5c7d75a860c20da0467"}
Jul 16 22:44:29.802908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:29.802521    2564 scope.go:117] "RemoveContainer" containerID="66e29a34955f42464b68fcf196b58430a554e88bea0881cbaaafdb3f23fd476d"
Jul 16 22:44:29.802908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:29.802817    2564 scope.go:117] "RemoveContainer" containerID="e04f47da6c0a3a36daf84394784de4cab97eba9fd9c0d5c7d75a860c20da0467"
Jul 16 22:44:29.803112 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:44:29.803089    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2_odh-ai-gateway-infra(24b35a63-1546-4fe4-8271-c220a83a5267)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" podUID="24b35a63-1546-4fe4-8271-c220a83a5267"
Jul 16 22:44:29.830179 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:29.830152    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"]
Jul 16 22:44:30.942564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:30.942538    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:31.117902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.117802    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b35a63-1546-4fe4-8271-c220a83a5267-maas-api-service-ca\") pod \"24b35a63-1546-4fe4-8271-c220a83a5267\" (UID: \"24b35a63-1546-4fe4-8271-c220a83a5267\") "
Jul 16 22:44:31.118228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.118205    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b35a63-1546-4fe4-8271-c220a83a5267-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24b35a63-1546-4fe4-8271-c220a83a5267" (UID: "24b35a63-1546-4fe4-8271-c220a83a5267"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:44:31.219517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.219483    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b35a63-1546-4fe4-8271-c220a83a5267-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:44:31.815188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.815156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2" event={"ID":"24b35a63-1546-4fe4-8271-c220a83a5267","Type":"ContainerDied","Data":"94054c9ad2023c540b864cda2eddf19e97c092562e1c1a8b0c97d8ca4e821efb"}
Jul 16 22:44:31.815188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.815185    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"
Jul 16 22:44:31.815443 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.815219    2564 scope.go:117] "RemoveContainer" containerID="e04f47da6c0a3a36daf84394784de4cab97eba9fd9c0d5c7d75a860c20da0467"
Jul 16 22:44:31.834879 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.834854    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"]
Jul 16 22:44:31.839161 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:31.839127    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mnwl2"]
Jul 16 22:44:33.601395 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:44:33.601364    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" path="/var/lib/kubelet/pods/24b35a63-1546-4fe4-8271-c220a83a5267/volumes"
Jul 16 22:45:00.139067 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139026    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"]
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139398    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139410    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139419    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139425    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139431    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139437    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="134acb00-0f05-4e19-bbd5-f8801b8052f8" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139496    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.139599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.139504    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.142445 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.142423    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"]
Jul 16 22:45:00.142591 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.142552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"
Jul 16 22:45:00.142721 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.142708    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.142721 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.142721    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.142842 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.142785    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b35a63-1546-4fe4-8271-c220a83a5267" containerName="revoke-keys"
Jul 16 22:45:00.144915 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.144896    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:45:00.145533 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.145519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"
Jul 16 22:45:00.149623 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.149603    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"]
Jul 16 22:45:00.153148 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.153106    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"]
Jul 16 22:45:00.155422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.155402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"
Jul 16 22:45:00.170698 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.170667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"
Jul 16 22:45:00.306071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.305963    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"]
Jul 16 22:45:00.307694 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:45:00.307644    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf575b11b_1bf4_4d8a_a855_299a1a3fecbe.slice/crio-3042728800e0fd1009f0b771313f74e8c35c5ebe5d6b901dc4c732787d626e0e WatchSource:0}: Error finding container 3042728800e0fd1009f0b771313f74e8c35c5ebe5d6b901dc4c732787d626e0e: Status 404 returned error can't find the container with id 3042728800e0fd1009f0b771313f74e8c35c5ebe5d6b901dc4c732787d626e0e
Jul 16 22:45:00.334024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.333996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"]
Jul 16 22:45:00.341326 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:45:00.341295    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4cd04f_8549_46a7_99f3_be220f8af99b.slice/crio-9e20df7b359217b1e19b50f9bf9dce12acafe03342844c78d6b32fc09eea12cc WatchSource:0}: Error finding container 9e20df7b359217b1e19b50f9bf9dce12acafe03342844c78d6b32fc09eea12cc: Status 404 returned error can't find the container with id 9e20df7b359217b1e19b50f9bf9dce12acafe03342844c78d6b32fc09eea12cc
Jul 16 22:45:00.975571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.975530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn" event={"ID":"f575b11b-1bf4-4d8a-a855-299a1a3fecbe","Type":"ContainerStarted","Data":"27b9156926d92b7b1cb52f74e1654699dd766514e247a6459bee95e0d71fb74b"}
Jul 16 22:45:00.975794 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.975592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn" event={"ID":"f575b11b-1bf4-4d8a-a855-299a1a3fecbe","Type":"ContainerStarted","Data":"3042728800e0fd1009f0b771313f74e8c35c5ebe5d6b901dc4c732787d626e0e"}
Jul 16 22:45:00.977099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.977067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerStarted","Data":"806f8572de06f7c291c573ad2f8959b4f0875872a96d1874f79a927571fcdb74"}
Jul 16 22:45:00.977250 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.977123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerStarted","Data":"9e20df7b359217b1e19b50f9bf9dce12acafe03342844c78d6b32fc09eea12cc"}
Jul 16 22:45:00.991168 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:00.991102    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn" podStartSLOduration=0.991088722 podStartE2EDuration="991.088722ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:45:00.990053838 +0000 UTC m=+1657.310609910" watchObservedRunningTime="2026-07-16 22:45:00.991088722 +0000 UTC m=+1657.311644794"
Jul 16 22:45:01.004892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:01.004849    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" podStartSLOduration=1.004835113 podStartE2EDuration="1.004835113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:45:01.004332184 +0000 UTC m=+1657.324888259" watchObservedRunningTime="2026-07-16 22:45:01.004835113 +0000 UTC m=+1657.325391186"
Jul 16 22:45:01.983794 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:01.983758    2564 generic.go:356] "Generic (PLEG): container finished" podID="f575b11b-1bf4-4d8a-a855-299a1a3fecbe" containerID="27b9156926d92b7b1cb52f74e1654699dd766514e247a6459bee95e0d71fb74b" exitCode=0
Jul 16 22:45:01.984279 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:01.983859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn" event={"ID":"f575b11b-1bf4-4d8a-a855-299a1a3fecbe","Type":"ContainerDied","Data":"27b9156926d92b7b1cb52f74e1654699dd766514e247a6459bee95e0d71fb74b"}
Jul 16 22:45:01.985316 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:01.985294    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerID="806f8572de06f7c291c573ad2f8959b4f0875872a96d1874f79a927571fcdb74" exitCode=7
Jul 16 22:45:01.985409 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:01.985375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerDied","Data":"806f8572de06f7c291c573ad2f8959b4f0875872a96d1874f79a927571fcdb74"}
Jul 16 22:45:01.985505 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:01.985493    2564 scope.go:117] "RemoveContainer" containerID="806f8572de06f7c291c573ad2f8959b4f0875872a96d1874f79a927571fcdb74"
Jul 16 22:45:02.992730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:02.992695    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerID="c24b5027960cc6ba6cd2409cf84766bd553b7f2370489dd3131738ec265b5625" exitCode=7
Jul 16 22:45:02.993195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:02.992786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerDied","Data":"c24b5027960cc6ba6cd2409cf84766bd553b7f2370489dd3131738ec265b5625"}
Jul 16 22:45:02.993195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:02.992853    2564 scope.go:117] "RemoveContainer" containerID="806f8572de06f7c291c573ad2f8959b4f0875872a96d1874f79a927571fcdb74"
Jul 16 22:45:02.993195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:02.992926    2564 scope.go:117] "RemoveContainer" containerID="c24b5027960cc6ba6cd2409cf84766bd553b7f2370489dd3131738ec265b5625"
Jul 16 22:45:02.993631 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:45:02.993209    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v_odh-ai-gateway-infra(1a4cd04f-8549-46a7-99f3-be220f8af99b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b"
Jul 16 22:45:03.128993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:03.128969    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"
Jul 16 22:45:03.999287 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:03.999256    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn"
Jul 16 22:45:03.999287 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:03.999280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-2rspn" event={"ID":"f575b11b-1bf4-4d8a-a855-299a1a3fecbe","Type":"ContainerDied","Data":"3042728800e0fd1009f0b771313f74e8c35c5ebe5d6b901dc4c732787d626e0e"}
Jul 16 22:45:03.999812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:03.999325    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3042728800e0fd1009f0b771313f74e8c35c5ebe5d6b901dc4c732787d626e0e"
Jul 16 22:45:04.001117 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:04.001095    2564 scope.go:117] "RemoveContainer" containerID="c24b5027960cc6ba6cd2409cf84766bd553b7f2370489dd3131738ec265b5625"
Jul 16 22:45:04.001396 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:45:04.001369    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v_odh-ai-gateway-infra(1a4cd04f-8549-46a7-99f3-be220f8af99b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b"
Jul 16 22:45:11.386263 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.386226    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"]
Jul 16 22:45:11.386624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.386567    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f575b11b-1bf4-4d8a-a855-299a1a3fecbe" containerName="cleanup"
Jul 16 22:45:11.386624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.386576    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f575b11b-1bf4-4d8a-a855-299a1a3fecbe" containerName="cleanup"
Jul 16 22:45:11.386710 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.386634    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f575b11b-1bf4-4d8a-a855-299a1a3fecbe" containerName="cleanup"
Jul 16 22:45:11.389681 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.389666    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:11.392019 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.392001    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:45:11.397754 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.397728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"]
Jul 16 22:45:11.469288 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.469253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b49ca4-8335-4470-83be-b58a55389229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p\" (UID: \"72b49ca4-8335-4470-83be-b58a55389229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:11.570579 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.570546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b49ca4-8335-4470-83be-b58a55389229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p\" (UID: \"72b49ca4-8335-4470-83be-b58a55389229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:11.571237 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.571217    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b49ca4-8335-4470-83be-b58a55389229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p\" (UID: \"72b49ca4-8335-4470-83be-b58a55389229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:11.701937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.701911    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:11.833550 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:11.833526    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"]
Jul 16 22:45:11.835651 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:45:11.835626    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b49ca4_8335_4470_83be_b58a55389229.slice/crio-9b8931d16d818d4662bf8e1797608a12f017876fe2231c538b1f3467a81bfa3f WatchSource:0}: Error finding container 9b8931d16d818d4662bf8e1797608a12f017876fe2231c538b1f3467a81bfa3f: Status 404 returned error can't find the container with id 9b8931d16d818d4662bf8e1797608a12f017876fe2231c538b1f3467a81bfa3f
Jul 16 22:45:12.048522 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:12.048481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" event={"ID":"72b49ca4-8335-4470-83be-b58a55389229","Type":"ContainerStarted","Data":"9b8931d16d818d4662bf8e1797608a12f017876fe2231c538b1f3467a81bfa3f"}
Jul 16 22:45:13.054915 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:13.054874    2564 generic.go:356] "Generic (PLEG): container finished" podID="72b49ca4-8335-4470-83be-b58a55389229" containerID="0eaa55eae350f5069e83e88d4b9b8034d8602b26203718c1df5c323f16313101" exitCode=7
Jul 16 22:45:13.055343 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:13.054956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" event={"ID":"72b49ca4-8335-4470-83be-b58a55389229","Type":"ContainerDied","Data":"0eaa55eae350f5069e83e88d4b9b8034d8602b26203718c1df5c323f16313101"}
Jul 16 22:45:13.055343 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:13.055168    2564 scope.go:117] "RemoveContainer" containerID="0eaa55eae350f5069e83e88d4b9b8034d8602b26203718c1df5c323f16313101"
Jul 16 22:45:14.062406 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:14.062376    2564 generic.go:356] "Generic (PLEG): container finished" podID="72b49ca4-8335-4470-83be-b58a55389229" containerID="cafceb413954c41f1220a9ca38c17f23dafa53d8e572ba6faee558e3fccb6b60" exitCode=7
Jul 16 22:45:14.062819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:14.062434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" event={"ID":"72b49ca4-8335-4470-83be-b58a55389229","Type":"ContainerDied","Data":"cafceb413954c41f1220a9ca38c17f23dafa53d8e572ba6faee558e3fccb6b60"}
Jul 16 22:45:14.062819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:14.062476    2564 scope.go:117] "RemoveContainer" containerID="0eaa55eae350f5069e83e88d4b9b8034d8602b26203718c1df5c323f16313101"
Jul 16 22:45:14.062819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:14.062757    2564 scope.go:117] "RemoveContainer" containerID="cafceb413954c41f1220a9ca38c17f23dafa53d8e572ba6faee558e3fccb6b60"
Jul 16 22:45:14.063014 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:45:14.062993    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p_odh-ai-gateway-infra(72b49ca4-8335-4470-83be-b58a55389229)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" podUID="72b49ca4-8335-4470-83be-b58a55389229"
Jul 16 22:45:15.070089 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:15.070057    2564 scope.go:117] "RemoveContainer" containerID="cafceb413954c41f1220a9ca38c17f23dafa53d8e572ba6faee558e3fccb6b60"
Jul 16 22:45:15.070520 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:45:15.070282    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p_odh-ai-gateway-infra(72b49ca4-8335-4470-83be-b58a55389229)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" podUID="72b49ca4-8335-4470-83be-b58a55389229"
Jul 16 22:45:17.596817 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:17.596787    2564 scope.go:117] "RemoveContainer" containerID="c24b5027960cc6ba6cd2409cf84766bd553b7f2370489dd3131738ec265b5625"
Jul 16 22:45:18.086652 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:18.086619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerStarted","Data":"1a58b180e24d6f991a2998c365f6d320f70a3eb64628ffc0784426ed9e09fd18"}
Jul 16 22:45:19.093597 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:19.093564    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerID="1a58b180e24d6f991a2998c365f6d320f70a3eb64628ffc0784426ed9e09fd18" exitCode=7
Jul 16 22:45:19.094086 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:19.093636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerDied","Data":"1a58b180e24d6f991a2998c365f6d320f70a3eb64628ffc0784426ed9e09fd18"}
Jul 16 22:45:19.094086 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:19.093693    2564 scope.go:117] "RemoveContainer" containerID="c24b5027960cc6ba6cd2409cf84766bd553b7f2370489dd3131738ec265b5625"
Jul 16 22:45:19.094086 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:19.093960    2564 scope.go:117] "RemoveContainer" containerID="1a58b180e24d6f991a2998c365f6d320f70a3eb64628ffc0784426ed9e09fd18"
Jul 16 22:45:19.094283 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:45:19.094250    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v_odh-ai-gateway-infra(1a4cd04f-8549-46a7-99f3-be220f8af99b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b"
Jul 16 22:45:19.117437 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:19.117404    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"]
Jul 16 22:45:20.240039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:20.240012    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"
Jul 16 22:45:21.111948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:21.111920    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"
Jul 16 22:45:21.111948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:21.111935    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v" event={"ID":"1a4cd04f-8549-46a7-99f3-be220f8af99b","Type":"ContainerDied","Data":"9e20df7b359217b1e19b50f9bf9dce12acafe03342844c78d6b32fc09eea12cc"}
Jul 16 22:45:21.112231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:21.111995    2564 scope.go:117] "RemoveContainer" containerID="1a58b180e24d6f991a2998c365f6d320f70a3eb64628ffc0784426ed9e09fd18"
Jul 16 22:45:21.132589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:21.132562    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"]
Jul 16 22:45:21.135726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:21.135702    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-5cb4a56a-29737365-t4j2v"]
Jul 16 22:45:21.601229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:21.601187    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" path="/var/lib/kubelet/pods/1a4cd04f-8549-46a7-99f3-be220f8af99b/volumes"
Jul 16 22:45:30.596013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:30.595984    2564 scope.go:117] "RemoveContainer" containerID="cafceb413954c41f1220a9ca38c17f23dafa53d8e572ba6faee558e3fccb6b60"
Jul 16 22:45:31.165312 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:31.165228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" event={"ID":"72b49ca4-8335-4470-83be-b58a55389229","Type":"ContainerStarted","Data":"b0731f5aa518407bb0769fda931cedc30f6d95544ffdee8591c155ed3b29d1e6"}
Jul 16 22:45:31.180910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:31.180865    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" podStartSLOduration=20.180847051 podStartE2EDuration="20.180847051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:45:31.179466631 +0000 UTC m=+1687.500022706" watchObservedRunningTime="2026-07-16 22:45:31.180847051 +0000 UTC m=+1687.501403177"
Jul 16 22:45:32.177384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:32.177347    2564 generic.go:356] "Generic (PLEG): container finished" podID="72b49ca4-8335-4470-83be-b58a55389229" containerID="b0731f5aa518407bb0769fda931cedc30f6d95544ffdee8591c155ed3b29d1e6" exitCode=7
Jul 16 22:45:32.177898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:32.177418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" event={"ID":"72b49ca4-8335-4470-83be-b58a55389229","Type":"ContainerDied","Data":"b0731f5aa518407bb0769fda931cedc30f6d95544ffdee8591c155ed3b29d1e6"}
Jul 16 22:45:32.177898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:32.177498    2564 scope.go:117] "RemoveContainer" containerID="cafceb413954c41f1220a9ca38c17f23dafa53d8e572ba6faee558e3fccb6b60"
Jul 16 22:45:32.177898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:32.177784    2564 scope.go:117] "RemoveContainer" containerID="b0731f5aa518407bb0769fda931cedc30f6d95544ffdee8591c155ed3b29d1e6"
Jul 16 22:45:32.178044 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:45:32.177980    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p_odh-ai-gateway-infra(72b49ca4-8335-4470-83be-b58a55389229)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" podUID="72b49ca4-8335-4470-83be-b58a55389229"
Jul 16 22:45:32.190519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:32.190492    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"]
Jul 16 22:45:33.317280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:33.317255    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:33.371219 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:33.371188    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b49ca4-8335-4470-83be-b58a55389229-maas-api-service-ca\") pod \"72b49ca4-8335-4470-83be-b58a55389229\" (UID: \"72b49ca4-8335-4470-83be-b58a55389229\") "
Jul 16 22:45:33.371590 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:33.371565    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72b49ca4-8335-4470-83be-b58a55389229-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72b49ca4-8335-4470-83be-b58a55389229" (UID: "72b49ca4-8335-4470-83be-b58a55389229"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:45:33.471893 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:33.471857    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b49ca4-8335-4470-83be-b58a55389229-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:45:34.190507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:34.190472    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"
Jul 16 22:45:34.190697 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:34.190511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p" event={"ID":"72b49ca4-8335-4470-83be-b58a55389229","Type":"ContainerDied","Data":"9b8931d16d818d4662bf8e1797608a12f017876fe2231c538b1f3467a81bfa3f"}
Jul 16 22:45:34.190697 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:34.190567    2564 scope.go:117] "RemoveContainer" containerID="b0731f5aa518407bb0769fda931cedc30f6d95544ffdee8591c155ed3b29d1e6"
Jul 16 22:45:34.206973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:34.206944    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"]
Jul 16 22:45:34.214804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:34.214772    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-4ld6p"]
Jul 16 22:45:35.602878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:45:35.602840    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72b49ca4-8335-4470-83be-b58a55389229" path="/var/lib/kubelet/pods/72b49ca4-8335-4470-83be-b58a55389229/volumes"
Jul 16 22:46:11.402746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.402711    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"]
Jul 16 22:46:11.403292 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.402920    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy" containerID="cri-o://68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d" gracePeriod=30
Jul 16 22:46:11.446830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.446794    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"]
Jul 16 22:46:11.447127 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447115    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:11.447127 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447129    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447153    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447159    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447173    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447178    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447190    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447195    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:11.447243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447246    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:11.447481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447253    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:11.447481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447262    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.447481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447269    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.447481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.447274    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:11.450280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.450264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:11.452530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.452507    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:46:11.452644 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.452507    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:46:11.458037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.458013    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"]
Jul 16 22:46:11.614701 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.614673    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98\" (UID: \"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:11.715572 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.715536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98\" (UID: \"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:11.716192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.716170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98\" (UID: \"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:11.761677 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.761641    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:11.891719 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:11.891693    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"]
Jul 16 22:46:11.893406 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:46:11.893381    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca8cd6c_7547_41d9_9d88_6951bdfc7a95.slice/crio-3805cbed80b46c94d2875952189346dfd833240d73f03babb904a933c5021bc9 WatchSource:0}: Error finding container 3805cbed80b46c94d2875952189346dfd833240d73f03babb904a933c5021bc9: Status 404 returned error can't find the container with id 3805cbed80b46c94d2875952189346dfd833240d73f03babb904a933c5021bc9
Jul 16 22:46:12.395239 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:12.395127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerStarted","Data":"e95c3bcc5f2c4b1c0de1b97b4111b1383c103603d70294288ff3b93909077edd"}
Jul 16 22:46:12.395239 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:12.395200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerStarted","Data":"3805cbed80b46c94d2875952189346dfd833240d73f03babb904a933c5021bc9"}
Jul 16 22:46:12.410582 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:12.410535    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" podStartSLOduration=1.410522067 podStartE2EDuration="1.410522067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:46:12.409348746 +0000 UTC m=+1728.729904824" watchObservedRunningTime="2026-07-16 22:46:12.410522067 +0000 UTC m=+1728.731078139"
Jul 16 22:46:13.401934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:13.401899    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerID="e95c3bcc5f2c4b1c0de1b97b4111b1383c103603d70294288ff3b93909077edd" exitCode=7
Jul 16 22:46:13.402157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:13.401988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerDied","Data":"e95c3bcc5f2c4b1c0de1b97b4111b1383c103603d70294288ff3b93909077edd"}
Jul 16 22:46:13.402157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:13.402120    2564 scope.go:117] "RemoveContainer" containerID="e95c3bcc5f2c4b1c0de1b97b4111b1383c103603d70294288ff3b93909077edd"
Jul 16 22:46:14.287757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:14.287662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:46:14.409514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:14.409480    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerID="a84139d4944e8a2c705beef9cd00e8210ce0779c4af7263d99de990176ff8ae5" exitCode=7
Jul 16 22:46:14.409514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:14.409518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerDied","Data":"a84139d4944e8a2c705beef9cd00e8210ce0779c4af7263d99de990176ff8ae5"}
Jul 16 22:46:14.409748 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:14.409560    2564 scope.go:117] "RemoveContainer" containerID="e95c3bcc5f2c4b1c0de1b97b4111b1383c103603d70294288ff3b93909077edd"
Jul 16 22:46:14.409814 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:14.409742    2564 scope.go:117] "RemoveContainer" containerID="a84139d4944e8a2c705beef9cd00e8210ce0779c4af7263d99de990176ff8ae5"
Jul 16 22:46:14.410007 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:14.409985    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98_odh-ai-gateway-infra(2ca8cd6c-7547-41d9-9d88-6951bdfc7a95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95"
Jul 16 22:46:15.416317 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:15.416234    2564 scope.go:117] "RemoveContainer" containerID="a84139d4944e8a2c705beef9cd00e8210ce0779c4af7263d99de990176ff8ae5"
Jul 16 22:46:15.416671 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:15.416437    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98_odh-ai-gateway-infra(2ca8cd6c-7547-41d9-9d88-6951bdfc7a95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95"
Jul 16 22:46:16.655627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.655599    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:46:16.661859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661644    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-podinfo\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.661859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661697    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-socket\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.661859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661730    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-envoy\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.661859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661769    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istiod-ca-cert\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.661859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-token\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.661859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661852    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-data\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.662299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.661931    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-certs\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.662299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knxmv\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-kube-api-access-knxmv\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.662299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662051    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-credential-socket\") pod \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\" (UID: \"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8\") "
Jul 16 22:46:16.662299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662154    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:46:16.662299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662212    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:16.662553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:46:16.662553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662447    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.662553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662467    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.662553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662507    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:46:16.662885 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.662853    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:46:16.664618 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.664592    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 22:46:16.664748 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.664731    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-kube-api-access-knxmv" (OuterVolumeSpecName: "kube-api-access-knxmv") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "kube-api-access-knxmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:46:16.664808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.664764    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:46:16.664808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.664765    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" (UID: "b7e292c2-e67b-4d0b-ae99-365d2d1da5f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:46:16.763719 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763688    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.763719 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763718    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.763907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763728    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knxmv\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-kube-api-access-knxmv\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.763907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763739    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.763907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763749    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.763907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763758    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:16.763907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:16.763765    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:17.427957 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.427920    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerID="68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d" exitCode=0
Jul 16 22:46:17.427957 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.427960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" event={"ID":"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8","Type":"ContainerDied","Data":"68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d"}
Jul 16 22:46:17.428251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.427966    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"
Jul 16 22:46:17.428251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.428012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr" event={"ID":"b7e292c2-e67b-4d0b-ae99-365d2d1da5f8","Type":"ContainerDied","Data":"6ce18b8de8693de2af505d9a85e048f83c48407f5d387702249aed71372ddae8"}
Jul 16 22:46:17.428251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.428039    2564 scope.go:117] "RemoveContainer" containerID="68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d"
Jul 16 22:46:17.437740 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.437724    2564 scope.go:117] "RemoveContainer" containerID="41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"
Jul 16 22:46:17.445858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.445840    2564 scope.go:117] "RemoveContainer" containerID="68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d"
Jul 16 22:46:17.446111 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:17.446092    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d\": container with ID starting with 68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d not found: ID does not exist" containerID="68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d"
Jul 16 22:46:17.446200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.446124    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d"} err="failed to get container status \"68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d\": rpc error: code = NotFound desc = could not find container \"68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d\": container with ID starting with 68d862dfa69d4451baf846aae11294ec172cf69ab74d703fd3511d3a39b8b76d not found: ID does not exist"
Jul 16 22:46:17.446200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.446164    2564 scope.go:117] "RemoveContainer" containerID="41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"
Jul 16 22:46:17.446405 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:17.446385    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2\": container with ID starting with 41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2 not found: ID does not exist" containerID="41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"
Jul 16 22:46:17.446448 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.446412    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2"} err="failed to get container status \"41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2\": rpc error: code = NotFound desc = could not find container \"41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2\": container with ID starting with 41fd0af208a0afabcbffe4a612dead8f8241c6a81050c01f48f6816773c425b2 not found: ID does not exist"
Jul 16 22:46:17.477352 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.477324    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"]
Jul 16 22:46:17.491652 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.491624    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5cb4a56a-openshift-default-58c6846d77-rpzrr"]
Jul 16 22:46:17.601786 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:17.601751    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" path="/var/lib/kubelet/pods/b7e292c2-e67b-4d0b-ae99-365d2d1da5f8/volumes"
Jul 16 22:46:23.734015 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.733981    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"]
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734338    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734350    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734361    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734366    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734373    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734379    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734389    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:23.734398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734394    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b49ca4-8335-4470-83be-b58a55389229" containerName="revoke-keys"
Jul 16 22:46:23.734661 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734449    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy"
Jul 16 22:46:23.734661 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.734456    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4cd04f-8549-46a7-99f3-be220f8af99b" containerName="cleanup"
Jul 16 22:46:23.739522 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.739504    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.742020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.741999    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-bf38b381-gw-openshift-default-dockercfg-6srtd\""
Jul 16 22:46:23.752147 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.752111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"]
Jul 16 22:46:23.827597 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-istio-token\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827793 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6z79\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-kube-api-access-s6z79\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827793 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97e99ac6-888f-468e-a324-f7fe5367b103-istiod-ca-cert\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827793 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-certs\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827793 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827745    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-socket\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827793 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-credential-socket\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-data\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827887    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-envoy\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.827987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.827914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97e99ac6-888f-468e-a324-f7fe5367b103-istio-podinfo\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-credential-socket\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-data\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-envoy\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97e99ac6-888f-468e-a324-f7fe5367b103-istio-podinfo\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-istio-token\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6z79\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-kube-api-access-s6z79\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97e99ac6-888f-468e-a324-f7fe5367b103-istiod-ca-cert\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-certs\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-socket\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-credential-socket\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929783 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929656    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-certs\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929783 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929709    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-socket\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.929861 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929815    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-data\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.930011 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.929992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97e99ac6-888f-468e-a324-f7fe5367b103-istiod-ca-cert\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.931646 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.931623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-envoy\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.931646 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.931641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97e99ac6-888f-468e-a324-f7fe5367b103-istio-podinfo\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.937384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.937358    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-istio-token\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:23.937469 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:23.937436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6z79\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-kube-api-access-s6z79\") pod \"e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") " pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:24.049722 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.049629    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:24.190008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.189885    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"]
Jul 16 22:46:24.192193 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:46:24.192163    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e99ac6_888f_468e_a324_f7fe5367b103.slice/crio-0bcb7e9d3734be11a3aac86750f52daaf9a11b431ee7fc56b8a7907388896e9c WatchSource:0}: Error finding container 0bcb7e9d3734be11a3aac86750f52daaf9a11b431ee7fc56b8a7907388896e9c: Status 404 returned error can't find the container with id 0bcb7e9d3734be11a3aac86750f52daaf9a11b431ee7fc56b8a7907388896e9c
Jul 16 22:46:24.194196 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.194166    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:46:24.194280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.194225    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:46:24.194280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.194252    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:46:24.465386 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.465348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" event={"ID":"97e99ac6-888f-468e-a324-f7fe5367b103","Type":"ContainerStarted","Data":"1ab998294874cd95f667b1b4394f5be420da24266efd784cb44444279bec72cc"}
Jul 16 22:46:24.465600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.465404    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" event={"ID":"97e99ac6-888f-468e-a324-f7fe5367b103","Type":"ContainerStarted","Data":"0bcb7e9d3734be11a3aac86750f52daaf9a11b431ee7fc56b8a7907388896e9c"}
Jul 16 22:46:24.489910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:24.489858    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" podStartSLOduration=1.489844131 podStartE2EDuration="1.489844131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:46:24.488102795 +0000 UTC m=+1740.808658865" watchObservedRunningTime="2026-07-16 22:46:24.489844131 +0000 UTC m=+1740.810400181"
Jul 16 22:46:25.050750 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:25.050709    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:25.055764 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:25.055738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:25.470276 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:25.470248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:25.471235 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:25.471215    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:27.882360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:27.882323    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:46:28.597147 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:28.597086    2564 scope.go:117] "RemoveContainer" containerID="a84139d4944e8a2c705beef9cd00e8210ce0779c4af7263d99de990176ff8ae5"
Jul 16 22:46:29.503045 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.503015    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerStarted","Data":"3be7b6f586fdb258d90862b11c520c9a44bbc7bf52eff94d2f7ddfc9e9e353a5"}
Jul 16 22:46:29.507027 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.506995    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"]
Jul 16 22:46:29.507480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.507463    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e292c2-e67b-4d0b-ae99-365d2d1da5f8" containerName="istio-proxy"
Jul 16 22:46:29.510462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.510444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:29.518176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.517921    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"]
Jul 16 22:46:29.593653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.593604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ae1851-48fd-477c-bc67-beba880fc001-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d\" (UID: \"75ae1851-48fd-477c-bc67-beba880fc001\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:29.694715 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.694674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ae1851-48fd-477c-bc67-beba880fc001-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d\" (UID: \"75ae1851-48fd-477c-bc67-beba880fc001\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:29.695282 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.695261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ae1851-48fd-477c-bc67-beba880fc001-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d\" (UID: \"75ae1851-48fd-477c-bc67-beba880fc001\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:29.859676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:29.859575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:30.033051 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.033004    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"]
Jul 16 22:46:30.034755 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:46:30.034724    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ae1851_48fd_477c_bc67_beba880fc001.slice/crio-0c825218d8cf8f217992285678057c33d748c7b9c785b228ccc0a561fd7a6d2a WatchSource:0}: Error finding container 0c825218d8cf8f217992285678057c33d748c7b9c785b228ccc0a561fd7a6d2a: Status 404 returned error can't find the container with id 0c825218d8cf8f217992285678057c33d748c7b9c785b228ccc0a561fd7a6d2a
Jul 16 22:46:30.509956 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.509917    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" event={"ID":"75ae1851-48fd-477c-bc67-beba880fc001","Type":"ContainerStarted","Data":"327c9b99c3c7ca4a50df46b62375942803b60573e22fa0cede6421e644ce6388"}
Jul 16 22:46:30.510398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.509972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" event={"ID":"75ae1851-48fd-477c-bc67-beba880fc001","Type":"ContainerStarted","Data":"0c825218d8cf8f217992285678057c33d748c7b9c785b228ccc0a561fd7a6d2a"}
Jul 16 22:46:30.511837 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.511813    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerID="3be7b6f586fdb258d90862b11c520c9a44bbc7bf52eff94d2f7ddfc9e9e353a5" exitCode=6
Jul 16 22:46:30.511948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.511865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerDied","Data":"3be7b6f586fdb258d90862b11c520c9a44bbc7bf52eff94d2f7ddfc9e9e353a5"}
Jul 16 22:46:30.511948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.511905    2564 scope.go:117] "RemoveContainer" containerID="a84139d4944e8a2c705beef9cd00e8210ce0779c4af7263d99de990176ff8ae5"
Jul 16 22:46:30.512213 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.512197    2564 scope.go:117] "RemoveContainer" containerID="3be7b6f586fdb258d90862b11c520c9a44bbc7bf52eff94d2f7ddfc9e9e353a5"
Jul 16 22:46:30.512406 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:30.512387    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98_odh-ai-gateway-infra(2ca8cd6c-7547-41d9-9d88-6951bdfc7a95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95"
Jul 16 22:46:30.528500 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.528474    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"]
Jul 16 22:46:30.531400 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:30.531356    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" podStartSLOduration=1.5313436280000001 podStartE2EDuration="1.531343628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:46:30.530774651 +0000 UTC m=+1746.851330725" watchObservedRunningTime="2026-07-16 22:46:30.531343628 +0000 UTC m=+1746.851899699"
Jul 16 22:46:31.518983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.518946    2564 generic.go:356] "Generic (PLEG): container finished" podID="75ae1851-48fd-477c-bc67-beba880fc001" containerID="327c9b99c3c7ca4a50df46b62375942803b60573e22fa0cede6421e644ce6388" exitCode=7
Jul 16 22:46:31.519408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.519028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" event={"ID":"75ae1851-48fd-477c-bc67-beba880fc001","Type":"ContainerDied","Data":"327c9b99c3c7ca4a50df46b62375942803b60573e22fa0cede6421e644ce6388"}
Jul 16 22:46:31.519408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.519236    2564 scope.go:117] "RemoveContainer" containerID="327c9b99c3c7ca4a50df46b62375942803b60573e22fa0cede6421e644ce6388"
Jul 16 22:46:31.684589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.684556    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:31.816763 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.816729    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95-maas-api-service-ca\") pod \"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95\" (UID: \"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95\") "
Jul 16 22:46:31.817084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.817057    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" (UID: "2ca8cd6c-7547-41d9-9d88-6951bdfc7a95"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:31.918310 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:31.918268    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:32.527837 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.527802    2564 generic.go:356] "Generic (PLEG): container finished" podID="75ae1851-48fd-477c-bc67-beba880fc001" containerID="c842572c34b05e74f8644929436f0e892b6bd41707bb037e99080a8d16d86291" exitCode=7
Jul 16 22:46:32.528356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.527894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" event={"ID":"75ae1851-48fd-477c-bc67-beba880fc001","Type":"ContainerDied","Data":"c842572c34b05e74f8644929436f0e892b6bd41707bb037e99080a8d16d86291"}
Jul 16 22:46:32.528356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.527962    2564 scope.go:117] "RemoveContainer" containerID="327c9b99c3c7ca4a50df46b62375942803b60573e22fa0cede6421e644ce6388"
Jul 16 22:46:32.528356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.528089    2564 scope.go:117] "RemoveContainer" containerID="c842572c34b05e74f8644929436f0e892b6bd41707bb037e99080a8d16d86291"
Jul 16 22:46:32.528356 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:32.528337    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d_odh-ai-gateway-infra(75ae1851-48fd-477c-bc67-beba880fc001)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" podUID="75ae1851-48fd-477c-bc67-beba880fc001"
Jul 16 22:46:32.529302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.529269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98" event={"ID":"2ca8cd6c-7547-41d9-9d88-6951bdfc7a95","Type":"ContainerDied","Data":"3805cbed80b46c94d2875952189346dfd833240d73f03babb904a933c5021bc9"}
Jul 16 22:46:32.529302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.529277    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"
Jul 16 22:46:32.539702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.539684    2564 scope.go:117] "RemoveContainer" containerID="3be7b6f586fdb258d90862b11c520c9a44bbc7bf52eff94d2f7ddfc9e9e353a5"
Jul 16 22:46:32.558563 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.558535    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"]
Jul 16 22:46:32.563858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:32.563833    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-k5z98"]
Jul 16 22:46:33.537649 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:33.537622    2564 scope.go:117] "RemoveContainer" containerID="c842572c34b05e74f8644929436f0e892b6bd41707bb037e99080a8d16d86291"
Jul 16 22:46:33.538076 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:33.537885    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d_odh-ai-gateway-infra(75ae1851-48fd-477c-bc67-beba880fc001)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" podUID="75ae1851-48fd-477c-bc67-beba880fc001"
Jul 16 22:46:33.603737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:33.603703    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" path="/var/lib/kubelet/pods/2ca8cd6c-7547-41d9-9d88-6951bdfc7a95/volumes"
Jul 16 22:46:35.552527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:35.552417    2564 generic.go:356] "Generic (PLEG): container finished" podID="97e99ac6-888f-468e-a324-f7fe5367b103" containerID="1ab998294874cd95f667b1b4394f5be420da24266efd784cb44444279bec72cc" exitCode=0
Jul 16 22:46:35.552527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:35.552499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" event={"ID":"97e99ac6-888f-468e-a324-f7fe5367b103","Type":"ContainerDied","Data":"1ab998294874cd95f667b1b4394f5be420da24266efd784cb44444279bec72cc"}
Jul 16 22:46:35.553095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:35.553040    2564 scope.go:117] "RemoveContainer" containerID="1ab998294874cd95f667b1b4394f5be420da24266efd784cb44444279bec72cc"
Jul 16 22:46:35.554041 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:35.554016    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:46:35.554101 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:35.554085    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:46:35.554152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:35.554116    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:46:36.560099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:36.560064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" event={"ID":"97e99ac6-888f-468e-a324-f7fe5367b103","Type":"ContainerStarted","Data":"f6c662ae5d8c82c0de161ac604713004e8ff7c64e3e940d5c9106bba011fa0fc"}
Jul 16 22:46:37.049967 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:37.049932    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:37.051215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:37.051186    2564 patch_prober.go:28] interesting pod/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 16 22:46:37.051359 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:37.051243    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 16 22:46:38.050970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:38.050925    2564 patch_prober.go:28] interesting pod/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 16 22:46:38.051428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:38.050989    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 16 22:46:39.050729 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:39.050690    2564 patch_prober.go:28] interesting pod/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 16 22:46:39.050916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:39.050747    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 16 22:46:40.050249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:40.050212    2564 patch_prober.go:28] interesting pod/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 16 22:46:40.050654 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:40.050274    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 16 22:46:40.471208 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:40.471174    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:41.054372 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:41.054337    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:41.055381 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:41.055360    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:46:48.596519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:48.596485    2564 scope.go:117] "RemoveContainer" containerID="c842572c34b05e74f8644929436f0e892b6bd41707bb037e99080a8d16d86291"
Jul 16 22:46:49.632811 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:49.632776    2564 generic.go:356] "Generic (PLEG): container finished" podID="75ae1851-48fd-477c-bc67-beba880fc001" containerID="cccdcaedf4def14146e1a18c296d0c14173e238d3897cd647ec981913a26407e" exitCode=7
Jul 16 22:46:49.633329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:49.632847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" event={"ID":"75ae1851-48fd-477c-bc67-beba880fc001","Type":"ContainerDied","Data":"cccdcaedf4def14146e1a18c296d0c14173e238d3897cd647ec981913a26407e"}
Jul 16 22:46:49.633329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:49.632905    2564 scope.go:117] "RemoveContainer" containerID="c842572c34b05e74f8644929436f0e892b6bd41707bb037e99080a8d16d86291"
Jul 16 22:46:49.633329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:49.633241    2564 scope.go:117] "RemoveContainer" containerID="cccdcaedf4def14146e1a18c296d0c14173e238d3897cd647ec981913a26407e"
Jul 16 22:46:49.633537 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:46:49.633515    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d_odh-ai-gateway-infra(75ae1851-48fd-477c-bc67-beba880fc001)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" podUID="75ae1851-48fd-477c-bc67-beba880fc001"
Jul 16 22:46:50.659200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:50.659167    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"]
Jul 16 22:46:50.793280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:50.793254    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:50.896417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:50.896384    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ae1851-48fd-477c-bc67-beba880fc001-maas-api-service-ca\") pod \"75ae1851-48fd-477c-bc67-beba880fc001\" (UID: \"75ae1851-48fd-477c-bc67-beba880fc001\") "
Jul 16 22:46:50.896774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:50.896753    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ae1851-48fd-477c-bc67-beba880fc001-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "75ae1851-48fd-477c-bc67-beba880fc001" (UID: "75ae1851-48fd-477c-bc67-beba880fc001"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:50.997125 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:50.997091    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ae1851-48fd-477c-bc67-beba880fc001-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:46:51.652274 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:51.652246    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"
Jul 16 22:46:51.652425 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:51.652285    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d" event={"ID":"75ae1851-48fd-477c-bc67-beba880fc001","Type":"ContainerDied","Data":"0c825218d8cf8f217992285678057c33d748c7b9c785b228ccc0a561fd7a6d2a"}
Jul 16 22:46:51.652425 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:51.652359    2564 scope.go:117] "RemoveContainer" containerID="cccdcaedf4def14146e1a18c296d0c14173e238d3897cd647ec981913a26407e"
Jul 16 22:46:51.670761 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:51.670712    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"]
Jul 16 22:46:51.673164 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:51.673122    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-5vh5d"]
Jul 16 22:46:53.602054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:46:53.602023    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75ae1851-48fd-477c-bc67-beba880fc001" path="/var/lib/kubelet/pods/75ae1851-48fd-477c-bc67-beba880fc001/volumes"
Jul 16 22:47:11.493951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.493911    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"]
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494280    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494291    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494300    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494305    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494312    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494318    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494327    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494348    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494368    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494374    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494423    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494429    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494435    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494442    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:11.494608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.494447    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:11.497054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.497038    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:11.499364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.499345    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:47:11.499469 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.499424    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:47:11.504881 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.504742    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"]
Jul 16 22:47:11.572040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.571998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838afdc-e3d6-42ce-9d0e-c61c227c025a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9\" (UID: \"3838afdc-e3d6-42ce-9d0e-c61c227c025a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:11.672940 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.672898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838afdc-e3d6-42ce-9d0e-c61c227c025a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9\" (UID: \"3838afdc-e3d6-42ce-9d0e-c61c227c025a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:11.673612 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.673585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838afdc-e3d6-42ce-9d0e-c61c227c025a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9\" (UID: \"3838afdc-e3d6-42ce-9d0e-c61c227c025a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:11.808830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.808730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:11.941895 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:11.941863    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"]
Jul 16 22:47:11.943394 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:47:11.943368    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3838afdc_e3d6_42ce_9d0e_c61c227c025a.slice/crio-7fa508cc731b67a9137c79e788aba8d18a384fa889ba80528fe44617d64c2188 WatchSource:0}: Error finding container 7fa508cc731b67a9137c79e788aba8d18a384fa889ba80528fe44617d64c2188: Status 404 returned error can't find the container with id 7fa508cc731b67a9137c79e788aba8d18a384fa889ba80528fe44617d64c2188
Jul 16 22:47:12.763034 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:12.763002    2564 generic.go:356] "Generic (PLEG): container finished" podID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerID="4f055ce9ce2d1c89c7ed475775d8f19bf04a45d63d8619a9fd09e27fbd67ccbe" exitCode=6
Jul 16 22:47:12.763432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:12.763065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" event={"ID":"3838afdc-e3d6-42ce-9d0e-c61c227c025a","Type":"ContainerDied","Data":"4f055ce9ce2d1c89c7ed475775d8f19bf04a45d63d8619a9fd09e27fbd67ccbe"}
Jul 16 22:47:12.763432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:12.763102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" event={"ID":"3838afdc-e3d6-42ce-9d0e-c61c227c025a","Type":"ContainerStarted","Data":"7fa508cc731b67a9137c79e788aba8d18a384fa889ba80528fe44617d64c2188"}
Jul 16 22:47:12.763432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:12.763283    2564 scope.go:117] "RemoveContainer" containerID="4f055ce9ce2d1c89c7ed475775d8f19bf04a45d63d8619a9fd09e27fbd67ccbe"
Jul 16 22:47:13.769906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:13.769817    2564 generic.go:356] "Generic (PLEG): container finished" podID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerID="a09b7e9259c062629365ac2494b1d5c485e58ffe305615c669936f090bdcbaf3" exitCode=6
Jul 16 22:47:13.770340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:13.769902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" event={"ID":"3838afdc-e3d6-42ce-9d0e-c61c227c025a","Type":"ContainerDied","Data":"a09b7e9259c062629365ac2494b1d5c485e58ffe305615c669936f090bdcbaf3"}
Jul 16 22:47:13.770340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:13.769958    2564 scope.go:117] "RemoveContainer" containerID="4f055ce9ce2d1c89c7ed475775d8f19bf04a45d63d8619a9fd09e27fbd67ccbe"
Jul 16 22:47:13.770340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:13.770083    2564 scope.go:117] "RemoveContainer" containerID="a09b7e9259c062629365ac2494b1d5c485e58ffe305615c669936f090bdcbaf3"
Jul 16 22:47:13.770340 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:13.770312    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9_odh-ai-gateway-infra(3838afdc-e3d6-42ce-9d0e-c61c227c025a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a"
Jul 16 22:47:14.776907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:14.776881    2564 scope.go:117] "RemoveContainer" containerID="a09b7e9259c062629365ac2494b1d5c485e58ffe305615c669936f090bdcbaf3"
Jul 16 22:47:14.777309 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:14.777090    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9_odh-ai-gateway-infra(3838afdc-e3d6-42ce-9d0e-c61c227c025a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a"
Jul 16 22:47:24.471587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:24.471554    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:47:24.478856 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:24.478835    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:47:29.558706 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.558670    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"]
Jul 16 22:47:29.559145 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.559050    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:29.559145 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.559062    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ae1851-48fd-477c-bc67-beba880fc001" containerName="revoke-keys"
Jul 16 22:47:29.559145 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.559117    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ca8cd6c-7547-41d9-9d88-6951bdfc7a95" containerName="revoke-keys"
Jul 16 22:47:29.561969 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.561952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:29.569562 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.569537    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"]
Jul 16 22:47:29.596739 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.596716    2564 scope.go:117] "RemoveContainer" containerID="a09b7e9259c062629365ac2494b1d5c485e58ffe305615c669936f090bdcbaf3"
Jul 16 22:47:29.636043 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.636009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20308c14-394c-4c69-a417-a7ff2c044071-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz\" (UID: \"20308c14-394c-4c69-a417-a7ff2c044071\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:29.637219 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.637197    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"]
Jul 16 22:47:29.637422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.637401    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy" containerID="cri-o://f6c662ae5d8c82c0de161ac604713004e8ff7c64e3e940d5c9106bba011fa0fc" gracePeriod=30
Jul 16 22:47:29.737422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.737380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20308c14-394c-4c69-a417-a7ff2c044071-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz\" (UID: \"20308c14-394c-4c69-a417-a7ff2c044071\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:29.737964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.737946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20308c14-394c-4c69-a417-a7ff2c044071-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz\" (UID: \"20308c14-394c-4c69-a417-a7ff2c044071\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:29.873341 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:29.873312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:30.008616 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.008591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"]
Jul 16 22:47:30.010617 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:47:30.010585    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20308c14_394c_4c69_a417_a7ff2c044071.slice/crio-442c0b3c36228d793fd17b1be60b198198b9f9a0f8442fe1f0f7088a6d444f5a WatchSource:0}: Error finding container 442c0b3c36228d793fd17b1be60b198198b9f9a0f8442fe1f0f7088a6d444f5a: Status 404 returned error can't find the container with id 442c0b3c36228d793fd17b1be60b198198b9f9a0f8442fe1f0f7088a6d444f5a
Jul 16 22:47:30.858417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.858386    2564 generic.go:356] "Generic (PLEG): container finished" podID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerID="dfadae0afb0809e3a68edf8e4428bed351c06e57210f2c74ebda360a5dc1c077" exitCode=6
Jul 16 22:47:30.858906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.858458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" event={"ID":"3838afdc-e3d6-42ce-9d0e-c61c227c025a","Type":"ContainerDied","Data":"dfadae0afb0809e3a68edf8e4428bed351c06e57210f2c74ebda360a5dc1c077"}
Jul 16 22:47:30.858906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.858527    2564 scope.go:117] "RemoveContainer" containerID="a09b7e9259c062629365ac2494b1d5c485e58ffe305615c669936f090bdcbaf3"
Jul 16 22:47:30.858906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.858857    2564 scope.go:117] "RemoveContainer" containerID="dfadae0afb0809e3a68edf8e4428bed351c06e57210f2c74ebda360a5dc1c077"
Jul 16 22:47:30.859091 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:30.859065    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9_odh-ai-gateway-infra(3838afdc-e3d6-42ce-9d0e-c61c227c025a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a"
Jul 16 22:47:30.860206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.860185    2564 generic.go:356] "Generic (PLEG): container finished" podID="20308c14-394c-4c69-a417-a7ff2c044071" containerID="fde5ee95b0058de10c5af212cf2838d5365c8b4983443b3d0f3f18783500b24f" exitCode=7
Jul 16 22:47:30.860293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.860218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" event={"ID":"20308c14-394c-4c69-a417-a7ff2c044071","Type":"ContainerDied","Data":"fde5ee95b0058de10c5af212cf2838d5365c8b4983443b3d0f3f18783500b24f"}
Jul 16 22:47:30.860293 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.860259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" event={"ID":"20308c14-394c-4c69-a417-a7ff2c044071","Type":"ContainerStarted","Data":"442c0b3c36228d793fd17b1be60b198198b9f9a0f8442fe1f0f7088a6d444f5a"}
Jul 16 22:47:30.860421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:30.860407    2564 scope.go:117] "RemoveContainer" containerID="fde5ee95b0058de10c5af212cf2838d5365c8b4983443b3d0f3f18783500b24f"
Jul 16 22:47:31.869688 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:31.869650    2564 generic.go:356] "Generic (PLEG): container finished" podID="20308c14-394c-4c69-a417-a7ff2c044071" containerID="bac3a19e992cc61d50b762d4260fb6ace046af7b73d8e4c94ddd67d6e476b925" exitCode=7
Jul 16 22:47:31.869688 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:31.869693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" event={"ID":"20308c14-394c-4c69-a417-a7ff2c044071","Type":"ContainerDied","Data":"bac3a19e992cc61d50b762d4260fb6ace046af7b73d8e4c94ddd67d6e476b925"}
Jul 16 22:47:31.870193 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:31.869748    2564 scope.go:117] "RemoveContainer" containerID="fde5ee95b0058de10c5af212cf2838d5365c8b4983443b3d0f3f18783500b24f"
Jul 16 22:47:31.870193 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:31.869895    2564 scope.go:117] "RemoveContainer" containerID="bac3a19e992cc61d50b762d4260fb6ace046af7b73d8e4c94ddd67d6e476b925"
Jul 16 22:47:31.870193 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:31.870119    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz_odh-ai-gateway-infra(20308c14-394c-4c69-a417-a7ff2c044071)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" podUID="20308c14-394c-4c69-a417-a7ff2c044071"
Jul 16 22:47:31.884855 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:31.884807    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"]
Jul 16 22:47:32.016173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.016123    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:32.160411 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.160319    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838afdc-e3d6-42ce-9d0e-c61c227c025a-maas-api-service-ca\") pod \"3838afdc-e3d6-42ce-9d0e-c61c227c025a\" (UID: \"3838afdc-e3d6-42ce-9d0e-c61c227c025a\") "
Jul 16 22:47:32.160698 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.160673    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3838afdc-e3d6-42ce-9d0e-c61c227c025a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3838afdc-e3d6-42ce-9d0e-c61c227c025a" (UID: "3838afdc-e3d6-42ce-9d0e-c61c227c025a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:47:32.261864 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.261830    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838afdc-e3d6-42ce-9d0e-c61c227c025a-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:32.484308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.484273    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:47:32.876861 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.876779    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"
Jul 16 22:47:32.876861 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.876814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9" event={"ID":"3838afdc-e3d6-42ce-9d0e-c61c227c025a","Type":"ContainerDied","Data":"7fa508cc731b67a9137c79e788aba8d18a384fa889ba80528fe44617d64c2188"}
Jul 16 22:47:32.877415 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.876882    2564 scope.go:117] "RemoveContainer" containerID="dfadae0afb0809e3a68edf8e4428bed351c06e57210f2c74ebda360a5dc1c077"
Jul 16 22:47:32.878911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.878854    2564 scope.go:117] "RemoveContainer" containerID="bac3a19e992cc61d50b762d4260fb6ace046af7b73d8e4c94ddd67d6e476b925"
Jul 16 22:47:32.879202 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:32.879173    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz_odh-ai-gateway-infra(20308c14-394c-4c69-a417-a7ff2c044071)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" podUID="20308c14-394c-4c69-a417-a7ff2c044071"
Jul 16 22:47:32.910116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.910084    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"]
Jul 16 22:47:32.913506 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:32.913479    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vqtd9"]
Jul 16 22:47:33.601895 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:33.601864    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" path="/var/lib/kubelet/pods/3838afdc-e3d6-42ce-9d0e-c61c227c025a/volumes"
Jul 16 22:47:34.892305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.892271    2564 generic.go:356] "Generic (PLEG): container finished" podID="97e99ac6-888f-468e-a324-f7fe5367b103" containerID="f6c662ae5d8c82c0de161ac604713004e8ff7c64e3e940d5c9106bba011fa0fc" exitCode=0
Jul 16 22:47:34.892653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.892343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" event={"ID":"97e99ac6-888f-468e-a324-f7fe5367b103","Type":"ContainerDied","Data":"f6c662ae5d8c82c0de161ac604713004e8ff7c64e3e940d5c9106bba011fa0fc"}
Jul 16 22:47:34.892653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.892416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng" event={"ID":"97e99ac6-888f-468e-a324-f7fe5367b103","Type":"ContainerDied","Data":"0bcb7e9d3734be11a3aac86750f52daaf9a11b431ee7fc56b8a7907388896e9c"}
Jul 16 22:47:34.892653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.892435    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bcb7e9d3734be11a3aac86750f52daaf9a11b431ee7fc56b8a7907388896e9c"
Jul 16 22:47:34.892653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.892449    2564 scope.go:117] "RemoveContainer" containerID="1ab998294874cd95f667b1b4394f5be420da24266efd784cb44444279bec72cc"
Jul 16 22:47:34.907909 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.907891    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:47:34.987200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987169    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97e99ac6-888f-468e-a324-f7fe5367b103-istio-podinfo\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-istio-token\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987236    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-certs\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-socket\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-envoy\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987316    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-credential-socket\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987338    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6z79\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-kube-api-access-s6z79\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987372    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97e99ac6-888f-468e-a324-f7fe5367b103-istiod-ca-cert\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987393    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-data\") pod \"97e99ac6-888f-468e-a324-f7fe5367b103\" (UID: \"97e99ac6-888f-468e-a324-f7fe5367b103\") "
Jul 16 22:47:34.987747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987637    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:47:34.987747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987651    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:47:34.987747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987629    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:47:34.987978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.987955    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e99ac6-888f-468e-a324-f7fe5367b103-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:47:34.988612 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.988581    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-data" (OuterVolumeSpecName: "istio-data") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:47:34.989882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.989850    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-istio-token" (OuterVolumeSpecName: "istio-token") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:47:34.990004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.989917    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:47:34.990004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.989916    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-kube-api-access-s6z79" (OuterVolumeSpecName: "kube-api-access-s6z79") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "kube-api-access-s6z79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:47:34.990118 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:34.990068    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97e99ac6-888f-468e-a324-f7fe5367b103-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97e99ac6-888f-468e-a324-f7fe5367b103" (UID: "97e99ac6-888f-468e-a324-f7fe5367b103"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 22:47:35.088261 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088228    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088261 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088260    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088272    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088283    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088296    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6z79\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-kube-api-access-s6z79\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088310    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97e99ac6-888f-468e-a324-f7fe5367b103-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088322    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97e99ac6-888f-468e-a324-f7fe5367b103-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088332    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97e99ac6-888f-468e-a324-f7fe5367b103-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.088535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.088345    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e99ac6-888f-468e-a324-f7fe5367b103-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.898998 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.898921    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"
Jul 16 22:47:35.935231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.935194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"]
Jul 16 22:47:35.939430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:35.939401    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-bf38b381-gw-openshift-default-5cbc8bd56c-tpdng"]
Jul 16 22:47:37.602409 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:37.602378    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" path="/var/lib/kubelet/pods/97e99ac6-888f-468e-a324-f7fe5367b103/volumes"
Jul 16 22:47:41.920045 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920011    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"]
Jul 16 22:47:41.920378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920365    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920378 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920377    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920384    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920392    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920405    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920411    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920420    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920425    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920431    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920436    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920488    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920497    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920503    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3838afdc-e3d6-42ce-9d0e-c61c227c025a" containerName="revoke-keys"
Jul 16 22:47:41.920712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.920509    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy"
Jul 16 22:47:41.924891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.924865    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:41.927780 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.927756    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-234edc78-openshift-default-dockercfg-4dsfz\""
Jul 16 22:47:41.937311 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:41.937286    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"]
Jul 16 22:47:42.058278 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-envoy\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058278 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-istio-token\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058483 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzhkr\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-kube-api-access-dzhkr\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058483 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-credential-socket\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058483 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058397    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-socket\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058483 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-data\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058483 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06da0dba-b829-4539-a577-c43a8a3f336d-istio-podinfo\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06da0dba-b829-4539-a577-c43a8a3f336d-istiod-ca-cert\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.058656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.058520    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-certs\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159017 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.158972    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-certs\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-envoy\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-istio-token\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzhkr\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-kube-api-access-dzhkr\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-credential-socket\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-socket\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159536 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-data\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159536 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06da0dba-b829-4539-a577-c43a8a3f336d-istio-podinfo\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159536 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06da0dba-b829-4539-a577-c43a8a3f336d-istiod-ca-cert\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159536 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159447    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-certs\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-credential-socket\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159594    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-socket\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.159737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.159699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-data\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.160129 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.160106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06da0dba-b829-4539-a577-c43a8a3f336d-istiod-ca-cert\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.161605 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.161579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-envoy\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.161832 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.161812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06da0dba-b829-4539-a577-c43a8a3f336d-istio-podinfo\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.168748 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.168681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzhkr\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-kube-api-access-dzhkr\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.168936 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.168912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-istio-token\") pod \"e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") " pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.237194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.237155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:42.378759 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.378695    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"]
Jul 16 22:47:42.381110 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:47:42.381079    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06da0dba_b829_4539_a577_c43a8a3f336d.slice/crio-af2df2f5743363a94397296e6dee81e35d8c36abe74d7c64d83b2d4e9fff1b54 WatchSource:0}: Error finding container af2df2f5743363a94397296e6dee81e35d8c36abe74d7c64d83b2d4e9fff1b54: Status 404 returned error can't find the container with id af2df2f5743363a94397296e6dee81e35d8c36abe74d7c64d83b2d4e9fff1b54
Jul 16 22:47:42.383355 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.383318    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:47:42.383423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.383406    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:47:42.383462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.383448    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:47:42.939118 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.939077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" event={"ID":"06da0dba-b829-4539-a577-c43a8a3f336d","Type":"ContainerStarted","Data":"6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"}
Jul 16 22:47:42.939528 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.939157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" event={"ID":"06da0dba-b829-4539-a577-c43a8a3f336d","Type":"ContainerStarted","Data":"af2df2f5743363a94397296e6dee81e35d8c36abe74d7c64d83b2d4e9fff1b54"}
Jul 16 22:47:42.961232 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:42.961171    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" podStartSLOduration=1.961122904 podStartE2EDuration="1.961122904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:47:42.959312821 +0000 UTC m=+1819.279868894" watchObservedRunningTime="2026-07-16 22:47:42.961122904 +0000 UTC m=+1819.281679044"
Jul 16 22:47:43.237344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:43.237310    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:43.242083 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:43.242056    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:43.944850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:43.944818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:43.945808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:43.945790    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:45.692440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:45.692404    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:47:46.596971 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:46.596934    2564 scope.go:117] "RemoveContainer" containerID="bac3a19e992cc61d50b762d4260fb6ace046af7b73d8e4c94ddd67d6e476b925"
Jul 16 22:47:47.966895 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:47.966858    2564 generic.go:356] "Generic (PLEG): container finished" podID="20308c14-394c-4c69-a417-a7ff2c044071" containerID="1aec51a5acec25ede1caa6af8a155c8878e1049c5d2a09c54fd576c9cc5d6b62" exitCode=6
Jul 16 22:47:47.967340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:47.966928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" event={"ID":"20308c14-394c-4c69-a417-a7ff2c044071","Type":"ContainerDied","Data":"1aec51a5acec25ede1caa6af8a155c8878e1049c5d2a09c54fd576c9cc5d6b62"}
Jul 16 22:47:47.967340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:47.966984    2564 scope.go:117] "RemoveContainer" containerID="bac3a19e992cc61d50b762d4260fb6ace046af7b73d8e4c94ddd67d6e476b925"
Jul 16 22:47:47.967340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:47.967288    2564 scope.go:117] "RemoveContainer" containerID="1aec51a5acec25ede1caa6af8a155c8878e1049c5d2a09c54fd576c9cc5d6b62"
Jul 16 22:47:47.967540 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:47.967517    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz_odh-ai-gateway-infra(20308c14-394c-4c69-a417-a7ff2c044071)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" podUID="20308c14-394c-4c69-a417-a7ff2c044071"
Jul 16 22:47:48.107672 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.107637    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"]
Jul 16 22:47:48.108036 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.108023    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="97e99ac6-888f-468e-a324-f7fe5367b103" containerName="istio-proxy"
Jul 16 22:47:48.112213 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.112196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:47:48.117820 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.117789    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"]
Jul 16 22:47:48.212996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.212958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ee581d5-5556-43c7-951b-8496786e2a51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-gdpsb\" (UID: \"2ee581d5-5556-43c7-951b-8496786e2a51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:47:48.313894 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.313804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ee581d5-5556-43c7-951b-8496786e2a51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-gdpsb\" (UID: \"2ee581d5-5556-43c7-951b-8496786e2a51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:47:48.314458 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.314440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ee581d5-5556-43c7-951b-8496786e2a51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-gdpsb\" (UID: \"2ee581d5-5556-43c7-951b-8496786e2a51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:47:48.424353 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.424318    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:47:48.595349 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.593382    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"]
Jul 16 22:47:48.598678 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:47:48.598643    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee581d5_5556_43c7_951b_8496786e2a51.slice/crio-0ecc7898458d6fd930fd844708c57886dc6a72674ae7e9937b3a7266b311b133 WatchSource:0}: Error finding container 0ecc7898458d6fd930fd844708c57886dc6a72674ae7e9937b3a7266b311b133: Status 404 returned error can't find the container with id 0ecc7898458d6fd930fd844708c57886dc6a72674ae7e9937b3a7266b311b133
Jul 16 22:47:48.984405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.984256    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerStarted","Data":"db12b938f2dacbd8e9d76933ab7a8ed0271ce2251a5a0ef69c8587c2d66b2b24"}
Jul 16 22:47:48.984405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.984365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerStarted","Data":"0ecc7898458d6fd930fd844708c57886dc6a72674ae7e9937b3a7266b311b133"}
Jul 16 22:47:48.994936 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:48.994888    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"]
Jul 16 22:47:49.003757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.003699    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" podStartSLOduration=1.003682156 podStartE2EDuration="1.003682156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:47:49.001489094 +0000 UTC m=+1825.322045167" watchObservedRunningTime="2026-07-16 22:47:49.003682156 +0000 UTC m=+1825.324238312"
Jul 16 22:47:49.143732 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.143670    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:49.221964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.221925    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20308c14-394c-4c69-a417-a7ff2c044071-maas-api-service-ca\") pod \"20308c14-394c-4c69-a417-a7ff2c044071\" (UID: \"20308c14-394c-4c69-a417-a7ff2c044071\") "
Jul 16 22:47:49.222329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.222304    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20308c14-394c-4c69-a417-a7ff2c044071-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20308c14-394c-4c69-a417-a7ff2c044071" (UID: "20308c14-394c-4c69-a417-a7ff2c044071"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:47:49.323049 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.323003    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20308c14-394c-4c69-a417-a7ff2c044071-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:47:49.994216 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.994174    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"
Jul 16 22:47:49.994717 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.994190    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz" event={"ID":"20308c14-394c-4c69-a417-a7ff2c044071","Type":"ContainerDied","Data":"442c0b3c36228d793fd17b1be60b198198b9f9a0f8442fe1f0f7088a6d444f5a"}
Jul 16 22:47:49.994717 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.994345    2564 scope.go:117] "RemoveContainer" containerID="1aec51a5acec25ede1caa6af8a155c8878e1049c5d2a09c54fd576c9cc5d6b62"
Jul 16 22:47:49.996040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.996013    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ee581d5-5556-43c7-951b-8496786e2a51" containerID="db12b938f2dacbd8e9d76933ab7a8ed0271ce2251a5a0ef69c8587c2d66b2b24" exitCode=7
Jul 16 22:47:49.996197 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.996175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerDied","Data":"db12b938f2dacbd8e9d76933ab7a8ed0271ce2251a5a0ef69c8587c2d66b2b24"}
Jul 16 22:47:49.996615 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:49.996378    2564 scope.go:117] "RemoveContainer" containerID="db12b938f2dacbd8e9d76933ab7a8ed0271ce2251a5a0ef69c8587c2d66b2b24"
Jul 16 22:47:50.026972 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:50.026941    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"]
Jul 16 22:47:50.031156 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:50.031117    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7rvpz"]
Jul 16 22:47:51.003396 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:51.003362    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ee581d5-5556-43c7-951b-8496786e2a51" containerID="eed1409fc078d5ce60e9ba11ed68b436845b68bb94c3660ab4f29242dd3992ef" exitCode=7
Jul 16 22:47:51.003396 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:51.003400    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerDied","Data":"eed1409fc078d5ce60e9ba11ed68b436845b68bb94c3660ab4f29242dd3992ef"}
Jul 16 22:47:51.003910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:51.003442    2564 scope.go:117] "RemoveContainer" containerID="db12b938f2dacbd8e9d76933ab7a8ed0271ce2251a5a0ef69c8587c2d66b2b24"
Jul 16 22:47:51.003910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:51.003670    2564 scope.go:117] "RemoveContainer" containerID="eed1409fc078d5ce60e9ba11ed68b436845b68bb94c3660ab4f29242dd3992ef"
Jul 16 22:47:51.004020 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:51.003918    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-gdpsb_odh-ai-gateway-infra(2ee581d5-5556-43c7-951b-8496786e2a51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" podUID="2ee581d5-5556-43c7-951b-8496786e2a51"
Jul 16 22:47:51.603407 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:51.603367    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20308c14-394c-4c69-a417-a7ff2c044071" path="/var/lib/kubelet/pods/20308c14-394c-4c69-a417-a7ff2c044071/volumes"
Jul 16 22:47:52.011519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:52.011493    2564 scope.go:117] "RemoveContainer" containerID="eed1409fc078d5ce60e9ba11ed68b436845b68bb94c3660ab4f29242dd3992ef"
Jul 16 22:47:52.011901 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:47:52.011679    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-gdpsb_odh-ai-gateway-infra(2ee581d5-5556-43c7-951b-8496786e2a51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" podUID="2ee581d5-5556-43c7-951b-8496786e2a51"
Jul 16 22:47:54.023331 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:54.023296    2564 generic.go:356] "Generic (PLEG): container finished" podID="06da0dba-b829-4539-a577-c43a8a3f336d" containerID="6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e" exitCode=0
Jul 16 22:47:54.023812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:54.023367    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" event={"ID":"06da0dba-b829-4539-a577-c43a8a3f336d","Type":"ContainerDied","Data":"6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"}
Jul 16 22:47:54.023812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:54.023753    2564 scope.go:117] "RemoveContainer" containerID="6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"
Jul 16 22:47:54.024752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:54.024724    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:47:54.024822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:54.024806    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:47:54.024868 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:54.024853    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:47:55.031241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:55.031204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" event={"ID":"06da0dba-b829-4539-a577-c43a8a3f336d","Type":"ContainerStarted","Data":"6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992"}
Jul 16 22:47:55.238115 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:55.238072    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:55.239376 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:55.239353    2564 patch_prober.go:28] interesting pod/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 16 22:47:55.239440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:55.239403    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 16 22:47:56.238426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:56.238379    2564 patch_prober.go:28] interesting pod/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 16 22:47:56.238924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:56.238442    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 16 22:47:57.238367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:57.238329    2564 patch_prober.go:28] interesting pod/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 16 22:47:57.238565 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:57.238387    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 16 22:47:58.237995 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:58.237954    2564 patch_prober.go:28] interesting pod/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 16 22:47:58.238215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:58.238015    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 16 22:47:58.945728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:58.945691    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:59.242544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:59.242456    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:47:59.243489 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:47:59.243465    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:48:05.598772 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:05.598737    2564 scope.go:117] "RemoveContainer" containerID="eed1409fc078d5ce60e9ba11ed68b436845b68bb94c3660ab4f29242dd3992ef"
Jul 16 22:48:06.094715 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:06.094677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerStarted","Data":"5d7ff95ce1f0e39c2111bf1a2293ba3ef594cf4ebcf5e0152c2169aae14cf934"}
Jul 16 22:48:07.101831 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:07.101797    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ee581d5-5556-43c7-951b-8496786e2a51" containerID="5d7ff95ce1f0e39c2111bf1a2293ba3ef594cf4ebcf5e0152c2169aae14cf934" exitCode=7
Jul 16 22:48:07.102295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:07.101874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerDied","Data":"5d7ff95ce1f0e39c2111bf1a2293ba3ef594cf4ebcf5e0152c2169aae14cf934"}
Jul 16 22:48:07.102295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:07.101935    2564 scope.go:117] "RemoveContainer" containerID="eed1409fc078d5ce60e9ba11ed68b436845b68bb94c3660ab4f29242dd3992ef"
Jul 16 22:48:07.102295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:07.102244    2564 scope.go:117] "RemoveContainer" containerID="5d7ff95ce1f0e39c2111bf1a2293ba3ef594cf4ebcf5e0152c2169aae14cf934"
Jul 16 22:48:07.102470 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:07.102450    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-gdpsb_odh-ai-gateway-infra(2ee581d5-5556-43c7-951b-8496786e2a51)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" podUID="2ee581d5-5556-43c7-951b-8496786e2a51"
Jul 16 22:48:07.125832 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:07.125795    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"]
Jul 16 22:48:08.242261 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:08.242235    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:48:08.295909 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:08.295881    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ee581d5-5556-43c7-951b-8496786e2a51-maas-api-service-ca\") pod \"2ee581d5-5556-43c7-951b-8496786e2a51\" (UID: \"2ee581d5-5556-43c7-951b-8496786e2a51\") "
Jul 16 22:48:08.296263 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:08.296241    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ee581d5-5556-43c7-951b-8496786e2a51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ee581d5-5556-43c7-951b-8496786e2a51" (UID: "2ee581d5-5556-43c7-951b-8496786e2a51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:48:08.397328 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:08.397226    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ee581d5-5556-43c7-951b-8496786e2a51-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:48:09.115053 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:09.115019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb" event={"ID":"2ee581d5-5556-43c7-951b-8496786e2a51","Type":"ContainerDied","Data":"0ecc7898458d6fd930fd844708c57886dc6a72674ae7e9937b3a7266b311b133"}
Jul 16 22:48:09.115053 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:09.115046    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"
Jul 16 22:48:09.115308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:09.115085    2564 scope.go:117] "RemoveContainer" containerID="5d7ff95ce1f0e39c2111bf1a2293ba3ef594cf4ebcf5e0152c2169aae14cf934"
Jul 16 22:48:09.139004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:09.138975    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"]
Jul 16 22:48:09.142782 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:09.142757    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-gdpsb"]
Jul 16 22:48:09.602064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:09.602028    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" path="/var/lib/kubelet/pods/2ee581d5-5556-43c7-951b-8496786e2a51/volumes"
Jul 16 22:48:11.536841 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.536807    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"]
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537145    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537157    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537164    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537170    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537183    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537191    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537202    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537207    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537219    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537224    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537276    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:11.537576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537284    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.537576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537288    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:11.537576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.537295    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:11.541718 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.541700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:11.544176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.544150    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:48:11.544315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.544181    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:48:11.549000 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.548971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"]
Jul 16 22:48:11.626237 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.626200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fa4c70-bd6a-4218-a155-dbae28fbdfeb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r\" (UID: \"41fa4c70-bd6a-4218-a155-dbae28fbdfeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:11.727432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.727392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fa4c70-bd6a-4218-a155-dbae28fbdfeb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r\" (UID: \"41fa4c70-bd6a-4218-a155-dbae28fbdfeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:11.728018 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.727999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fa4c70-bd6a-4218-a155-dbae28fbdfeb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r\" (UID: \"41fa4c70-bd6a-4218-a155-dbae28fbdfeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:11.854314 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.854219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:11.986309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:11.986282    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"]
Jul 16 22:48:11.987336 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:48:11.987309    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41fa4c70_bd6a_4218_a155_dbae28fbdfeb.slice/crio-4b1bdb22fa85954115a704bde650076dc4f7f22fea4040bfd9ea840a8bfa159d WatchSource:0}: Error finding container 4b1bdb22fa85954115a704bde650076dc4f7f22fea4040bfd9ea840a8bfa159d: Status 404 returned error can't find the container with id 4b1bdb22fa85954115a704bde650076dc4f7f22fea4040bfd9ea840a8bfa159d
Jul 16 22:48:12.132127 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:12.132044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" event={"ID":"41fa4c70-bd6a-4218-a155-dbae28fbdfeb","Type":"ContainerStarted","Data":"4b1bdb22fa85954115a704bde650076dc4f7f22fea4040bfd9ea840a8bfa159d"}
Jul 16 22:48:13.139037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:13.139001    2564 generic.go:356] "Generic (PLEG): container finished" podID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerID="17ea50275b95f13195b2f7b465e4e32770ea351e2cf345d243fe4ad3c073c208" exitCode=6
Jul 16 22:48:13.139426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:13.139049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" event={"ID":"41fa4c70-bd6a-4218-a155-dbae28fbdfeb","Type":"ContainerDied","Data":"17ea50275b95f13195b2f7b465e4e32770ea351e2cf345d243fe4ad3c073c208"}
Jul 16 22:48:13.139426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:13.139300    2564 scope.go:117] "RemoveContainer" containerID="17ea50275b95f13195b2f7b465e4e32770ea351e2cf345d243fe4ad3c073c208"
Jul 16 22:48:14.146026 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:14.145930    2564 generic.go:356] "Generic (PLEG): container finished" podID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerID="9037310a148e43058a539c9f4ff849d3d381bf99494d00a3e3d10f3bfafdacd1" exitCode=6
Jul 16 22:48:14.146446 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:14.146024    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" event={"ID":"41fa4c70-bd6a-4218-a155-dbae28fbdfeb","Type":"ContainerDied","Data":"9037310a148e43058a539c9f4ff849d3d381bf99494d00a3e3d10f3bfafdacd1"}
Jul 16 22:48:14.146446 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:14.146101    2564 scope.go:117] "RemoveContainer" containerID="17ea50275b95f13195b2f7b465e4e32770ea351e2cf345d243fe4ad3c073c208"
Jul 16 22:48:14.146446 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:14.146250    2564 scope.go:117] "RemoveContainer" containerID="9037310a148e43058a539c9f4ff849d3d381bf99494d00a3e3d10f3bfafdacd1"
Jul 16 22:48:14.146602 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:14.146500    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r_odh-ai-gateway-infra(41fa4c70-bd6a-4218-a155-dbae28fbdfeb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb"
Jul 16 22:48:15.155682 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:15.155652    2564 scope.go:117] "RemoveContainer" containerID="9037310a148e43058a539c9f4ff849d3d381bf99494d00a3e3d10f3bfafdacd1"
Jul 16 22:48:15.156048 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:15.155847    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r_odh-ai-gateway-infra(41fa4c70-bd6a-4218-a155-dbae28fbdfeb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb"
Jul 16 22:48:27.596459 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:27.596420    2564 scope.go:117] "RemoveContainer" containerID="9037310a148e43058a539c9f4ff849d3d381bf99494d00a3e3d10f3bfafdacd1"
Jul 16 22:48:28.223156 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:28.223105    2564 generic.go:356] "Generic (PLEG): container finished" podID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerID="2509dd410c8252190da02e472e5b1d02fd78493b8617bb65ccba4fd5625d8c2a" exitCode=6
Jul 16 22:48:28.223315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:28.223182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" event={"ID":"41fa4c70-bd6a-4218-a155-dbae28fbdfeb","Type":"ContainerDied","Data":"2509dd410c8252190da02e472e5b1d02fd78493b8617bb65ccba4fd5625d8c2a"}
Jul 16 22:48:28.223315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:28.223258    2564 scope.go:117] "RemoveContainer" containerID="9037310a148e43058a539c9f4ff849d3d381bf99494d00a3e3d10f3bfafdacd1"
Jul 16 22:48:28.223619 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:28.223600    2564 scope.go:117] "RemoveContainer" containerID="2509dd410c8252190da02e472e5b1d02fd78493b8617bb65ccba4fd5625d8c2a"
Jul 16 22:48:28.223873 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:28.223851    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r_odh-ai-gateway-infra(41fa4c70-bd6a-4218-a155-dbae28fbdfeb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb"
Jul 16 22:48:29.250488 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.250455    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"]
Jul 16 22:48:29.383535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.383505    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:29.493986 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.493951    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fa4c70-bd6a-4218-a155-dbae28fbdfeb-maas-api-service-ca\") pod \"41fa4c70-bd6a-4218-a155-dbae28fbdfeb\" (UID: \"41fa4c70-bd6a-4218-a155-dbae28fbdfeb\") "
Jul 16 22:48:29.494371 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.494348    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41fa4c70-bd6a-4218-a155-dbae28fbdfeb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41fa4c70-bd6a-4218-a155-dbae28fbdfeb" (UID: "41fa4c70-bd6a-4218-a155-dbae28fbdfeb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:48:29.595458 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.595361    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fa4c70-bd6a-4218-a155-dbae28fbdfeb-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:48:29.609725 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.609694    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"]
Jul 16 22:48:29.610099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610082    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.610099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610099    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610106    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610112    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610151    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610159    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610214    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ee581d5-5556-43c7-951b-8496786e2a51" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610220    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610227    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20308c14-394c-4c69-a417-a7ff2c044071" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610234    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.610242 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.610239    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:29.613066 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.613052    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:29.620501 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.620479    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"]
Jul 16 22:48:29.797875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.797830    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg\" (UID: \"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:29.899372 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.899275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg\" (UID: \"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:29.899902 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.899879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg\" (UID: \"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:29.924315 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:29.924273    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:30.056789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.056764    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"]
Jul 16 22:48:30.058659 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:48:30.058625    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecfe438_3b93_4305_8afe_4d6d1ceda5f5.slice/crio-740078e5f1e6967696075184c1af8488329b271a6b48a6b084ae05a9aa1e431b WatchSource:0}: Error finding container 740078e5f1e6967696075184c1af8488329b271a6b48a6b084ae05a9aa1e431b: Status 404 returned error can't find the container with id 740078e5f1e6967696075184c1af8488329b271a6b48a6b084ae05a9aa1e431b
Jul 16 22:48:30.236273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.236234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" event={"ID":"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5","Type":"ContainerStarted","Data":"740078e5f1e6967696075184c1af8488329b271a6b48a6b084ae05a9aa1e431b"}
Jul 16 22:48:30.237423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.237398    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r" event={"ID":"41fa4c70-bd6a-4218-a155-dbae28fbdfeb","Type":"ContainerDied","Data":"4b1bdb22fa85954115a704bde650076dc4f7f22fea4040bfd9ea840a8bfa159d"}
Jul 16 22:48:30.237528 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.237429    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"
Jul 16 22:48:30.237528 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.237442    2564 scope.go:117] "RemoveContainer" containerID="2509dd410c8252190da02e472e5b1d02fd78493b8617bb65ccba4fd5625d8c2a"
Jul 16 22:48:30.254329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.254306    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"]
Jul 16 22:48:30.260505 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:30.260483    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-lbn2r"]
Jul 16 22:48:31.243850 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:31.243815    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerID="260fe7536b9337f34d2f5b4e67ee3bd238549690907be20b48a3057fd7f0c1d2" exitCode=6
Jul 16 22:48:31.244050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:31.243899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" event={"ID":"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5","Type":"ContainerDied","Data":"260fe7536b9337f34d2f5b4e67ee3bd238549690907be20b48a3057fd7f0c1d2"}
Jul 16 22:48:31.244119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:31.244081    2564 scope.go:117] "RemoveContainer" containerID="260fe7536b9337f34d2f5b4e67ee3bd238549690907be20b48a3057fd7f0c1d2"
Jul 16 22:48:31.605076 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:31.605040    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" path="/var/lib/kubelet/pods/41fa4c70-bd6a-4218-a155-dbae28fbdfeb/volumes"
Jul 16 22:48:32.251601 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:32.251506    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerID="bbe1d06d94b1db740ce4f873a3fd13d5ae8a7a44f3caafbbda8a3a087ed30c9f" exitCode=6
Jul 16 22:48:32.251782 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:32.251596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" event={"ID":"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5","Type":"ContainerDied","Data":"bbe1d06d94b1db740ce4f873a3fd13d5ae8a7a44f3caafbbda8a3a087ed30c9f"}
Jul 16 22:48:32.251782 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:32.251666    2564 scope.go:117] "RemoveContainer" containerID="260fe7536b9337f34d2f5b4e67ee3bd238549690907be20b48a3057fd7f0c1d2"
Jul 16 22:48:32.251905 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:32.251892    2564 scope.go:117] "RemoveContainer" containerID="bbe1d06d94b1db740ce4f873a3fd13d5ae8a7a44f3caafbbda8a3a087ed30c9f"
Jul 16 22:48:32.252177 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:32.252151    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg_odh-ai-gateway-infra(4ecfe438-3b93-4305-8afe-4d6d1ceda5f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5"
Jul 16 22:48:33.258310 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:33.258285    2564 scope.go:117] "RemoveContainer" containerID="bbe1d06d94b1db740ce4f873a3fd13d5ae8a7a44f3caafbbda8a3a087ed30c9f"
Jul 16 22:48:33.258734 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:33.258495    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg_odh-ai-gateway-infra(4ecfe438-3b93-4305-8afe-4d6d1ceda5f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5"
Jul 16 22:48:44.596265 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:44.596232    2564 scope.go:117] "RemoveContainer" containerID="bbe1d06d94b1db740ce4f873a3fd13d5ae8a7a44f3caafbbda8a3a087ed30c9f"
Jul 16 22:48:45.322330 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:45.322298    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerID="9f1d00b34c57ef72d9f5f31a0ba79d53ee2da76f0b3cd268a99122f5c738af4c" exitCode=6
Jul 16 22:48:45.322503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:45.322366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" event={"ID":"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5","Type":"ContainerDied","Data":"9f1d00b34c57ef72d9f5f31a0ba79d53ee2da76f0b3cd268a99122f5c738af4c"}
Jul 16 22:48:45.322503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:45.322423    2564 scope.go:117] "RemoveContainer" containerID="bbe1d06d94b1db740ce4f873a3fd13d5ae8a7a44f3caafbbda8a3a087ed30c9f"
Jul 16 22:48:45.322714 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:45.322689    2564 scope.go:117] "RemoveContainer" containerID="9f1d00b34c57ef72d9f5f31a0ba79d53ee2da76f0b3cd268a99122f5c738af4c"
Jul 16 22:48:45.322952 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:45.322932    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg_odh-ai-gateway-infra(4ecfe438-3b93-4305-8afe-4d6d1ceda5f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5"
Jul 16 22:48:46.349585 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:46.349367    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"]
Jul 16 22:48:46.489989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:46.489963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:46.540290 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:46.540259    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5-maas-api-service-ca\") pod \"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5\" (UID: \"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5\") "
Jul 16 22:48:46.540653 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:46.540626    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" (UID: "4ecfe438-3b93-4305-8afe-4d6d1ceda5f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:48:46.641848 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:46.641749    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:48:47.335390 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:47.335349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg" event={"ID":"4ecfe438-3b93-4305-8afe-4d6d1ceda5f5","Type":"ContainerDied","Data":"740078e5f1e6967696075184c1af8488329b271a6b48a6b084ae05a9aa1e431b"}
Jul 16 22:48:47.335587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:47.335414    2564 scope.go:117] "RemoveContainer" containerID="9f1d00b34c57ef72d9f5f31a0ba79d53ee2da76f0b3cd268a99122f5c738af4c"
Jul 16 22:48:47.335587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:47.335370    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"
Jul 16 22:48:47.356844 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:47.356816    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"]
Jul 16 22:48:47.366075 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:47.366050    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-kr8gg"]
Jul 16 22:48:47.601900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:47.601804    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" path="/var/lib/kubelet/pods/4ecfe438-3b93-4305-8afe-4d6d1ceda5f5/volumes"
Jul 16 22:48:48.167548 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167513    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"]
Jul 16 22:48:48.167845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167833    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:48:48.167891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167847    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:48:48.167891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167864    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:48.167891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167872    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="41fa4c70-bd6a-4218-a155-dbae28fbdfeb" containerName="revoke-keys"
Jul 16 22:48:48.167891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167885    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:48:48.167891 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167891    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:48:48.168050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167941    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:48:48.168050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.167950    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:48:48.172270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.172251    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:48:48.174818 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.174796    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:48:48.175014 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.174987    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:48:48.178771 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.178751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"]
Jul 16 22:48:48.256503 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.256463    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f905ce42-5e1f-4572-89f1-8206da9486ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-5cx9t\" (UID: \"f905ce42-5e1f-4572-89f1-8206da9486ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:48:48.357188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.357119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f905ce42-5e1f-4572-89f1-8206da9486ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-5cx9t\" (UID: \"f905ce42-5e1f-4572-89f1-8206da9486ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:48:48.357728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.357708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f905ce42-5e1f-4572-89f1-8206da9486ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-5cx9t\" (UID: \"f905ce42-5e1f-4572-89f1-8206da9486ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:48:48.484185 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.484120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:48:48.612361 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:48.612331    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"]
Jul 16 22:48:48.612943 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:48:48.612924    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf905ce42_5e1f_4572_89f1_8206da9486ad.slice/crio-67de9b154c32ca0ee3e83d16ad36843ea93e943aa98f553ae01cdcbf5174b82d WatchSource:0}: Error finding container 67de9b154c32ca0ee3e83d16ad36843ea93e943aa98f553ae01cdcbf5174b82d: Status 404 returned error can't find the container with id 67de9b154c32ca0ee3e83d16ad36843ea93e943aa98f553ae01cdcbf5174b82d
Jul 16 22:48:49.348387 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:49.348356    2564 generic.go:356] "Generic (PLEG): container finished" podID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerID="efc4c412ed182d4d2d28ee8254dbe3c1587d5889e3596ea0d22b9b344d21005c" exitCode=7
Jul 16 22:48:49.348579 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:49.348440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" event={"ID":"f905ce42-5e1f-4572-89f1-8206da9486ad","Type":"ContainerDied","Data":"efc4c412ed182d4d2d28ee8254dbe3c1587d5889e3596ea0d22b9b344d21005c"}
Jul 16 22:48:49.348579 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:49.348490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" event={"ID":"f905ce42-5e1f-4572-89f1-8206da9486ad","Type":"ContainerStarted","Data":"67de9b154c32ca0ee3e83d16ad36843ea93e943aa98f553ae01cdcbf5174b82d"}
Jul 16 22:48:49.348702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:49.348593    2564 scope.go:117] "RemoveContainer" containerID="efc4c412ed182d4d2d28ee8254dbe3c1587d5889e3596ea0d22b9b344d21005c"
Jul 16 22:48:50.355181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:50.355071    2564 generic.go:356] "Generic (PLEG): container finished" podID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerID="6c6152211a48daa2df221399a4117d46356ecd009d342c2c5708d561390f7e24" exitCode=7
Jul 16 22:48:50.355181 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:50.355159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" event={"ID":"f905ce42-5e1f-4572-89f1-8206da9486ad","Type":"ContainerDied","Data":"6c6152211a48daa2df221399a4117d46356ecd009d342c2c5708d561390f7e24"}
Jul 16 22:48:50.355617 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:50.355202    2564 scope.go:117] "RemoveContainer" containerID="efc4c412ed182d4d2d28ee8254dbe3c1587d5889e3596ea0d22b9b344d21005c"
Jul 16 22:48:50.355617 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:50.355385    2564 scope.go:117] "RemoveContainer" containerID="6c6152211a48daa2df221399a4117d46356ecd009d342c2c5708d561390f7e24"
Jul 16 22:48:50.355728 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:50.355628    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-5cx9t_odh-ai-gateway-infra(f905ce42-5e1f-4572-89f1-8206da9486ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad"
Jul 16 22:48:51.361975 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:48:51.361946    2564 scope.go:117] "RemoveContainer" containerID="6c6152211a48daa2df221399a4117d46356ecd009d342c2c5708d561390f7e24"
Jul 16 22:48:51.362416 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:48:51.362126    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-5cx9t_odh-ai-gateway-infra(f905ce42-5e1f-4572-89f1-8206da9486ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad"
Jul 16 22:49:01.585614 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.585578    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"]
Jul 16 22:49:01.586021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.585915    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:49:01.586021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.585926    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:49:01.586021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.585998    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecfe438-3b93-4305-8afe-4d6d1ceda5f5" containerName="revoke-keys"
Jul 16 22:49:01.588776 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.588761    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:01.602326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.602301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"]
Jul 16 22:49:01.683812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.683761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4951d3b-f0bd-4243-9866-3a862410a7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb\" (UID: \"e4951d3b-f0bd-4243-9866-3a862410a7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:01.784516 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.784481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4951d3b-f0bd-4243-9866-3a862410a7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb\" (UID: \"e4951d3b-f0bd-4243-9866-3a862410a7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:01.785099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.785077    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4951d3b-f0bd-4243-9866-3a862410a7dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb\" (UID: \"e4951d3b-f0bd-4243-9866-3a862410a7dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:01.901251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:01.901169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:02.032759 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:02.032728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"]
Jul 16 22:49:02.034512 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:49:02.034484    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4951d3b_f0bd_4243_9866_3a862410a7dc.slice/crio-e2cb330bd09208ae802d54a24ef41e9ad7d659374b24da42ae8c6bd57b852812 WatchSource:0}: Error finding container e2cb330bd09208ae802d54a24ef41e9ad7d659374b24da42ae8c6bd57b852812: Status 404 returned error can't find the container with id e2cb330bd09208ae802d54a24ef41e9ad7d659374b24da42ae8c6bd57b852812
Jul 16 22:49:02.434693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:02.434649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" event={"ID":"e4951d3b-f0bd-4243-9866-3a862410a7dc","Type":"ContainerStarted","Data":"9edb0e4eef2736016e436bbbcbe76e283f1988205ec71831d76923543e6dae25"}
Jul 16 22:49:02.434901 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:02.434742    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" event={"ID":"e4951d3b-f0bd-4243-9866-3a862410a7dc","Type":"ContainerStarted","Data":"e2cb330bd09208ae802d54a24ef41e9ad7d659374b24da42ae8c6bd57b852812"}
Jul 16 22:49:02.452671 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:02.452609    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" podStartSLOduration=1.4525953600000001 podStartE2EDuration="1.45259536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:49:02.449695492 +0000 UTC m=+1898.770251565" watchObservedRunningTime="2026-07-16 22:49:02.45259536 +0000 UTC m=+1898.773151409"
Jul 16 22:49:03.441518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:03.441482    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerID="9edb0e4eef2736016e436bbbcbe76e283f1988205ec71831d76923543e6dae25" exitCode=6
Jul 16 22:49:03.441946 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:03.441571    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" event={"ID":"e4951d3b-f0bd-4243-9866-3a862410a7dc","Type":"ContainerDied","Data":"9edb0e4eef2736016e436bbbcbe76e283f1988205ec71831d76923543e6dae25"}
Jul 16 22:49:03.441946 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:03.441817    2564 scope.go:117] "RemoveContainer" containerID="9edb0e4eef2736016e436bbbcbe76e283f1988205ec71831d76923543e6dae25"
Jul 16 22:49:04.448562 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:04.448528    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerID="94273038198bca0127fa440963af4c104beeb55d21d77adbafb6bf1809db0e72" exitCode=6
Jul 16 22:49:04.449022 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:04.448607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" event={"ID":"e4951d3b-f0bd-4243-9866-3a862410a7dc","Type":"ContainerDied","Data":"94273038198bca0127fa440963af4c104beeb55d21d77adbafb6bf1809db0e72"}
Jul 16 22:49:04.449022 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:04.448663    2564 scope.go:117] "RemoveContainer" containerID="9edb0e4eef2736016e436bbbcbe76e283f1988205ec71831d76923543e6dae25"
Jul 16 22:49:04.449022 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:04.448858    2564 scope.go:117] "RemoveContainer" containerID="94273038198bca0127fa440963af4c104beeb55d21d77adbafb6bf1809db0e72"
Jul 16 22:49:04.449214 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:04.449108    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb_odh-ai-gateway-infra(e4951d3b-f0bd-4243-9866-3a862410a7dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc"
Jul 16 22:49:05.456410 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:05.456377    2564 scope.go:117] "RemoveContainer" containerID="94273038198bca0127fa440963af4c104beeb55d21d77adbafb6bf1809db0e72"
Jul 16 22:49:05.456774 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:05.456574    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb_odh-ai-gateway-infra(e4951d3b-f0bd-4243-9866-3a862410a7dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc"
Jul 16 22:49:05.599035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:05.599008    2564 scope.go:117] "RemoveContainer" containerID="6c6152211a48daa2df221399a4117d46356ecd009d342c2c5708d561390f7e24"
Jul 16 22:49:06.463639 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:06.463605    2564 generic.go:356] "Generic (PLEG): container finished" podID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerID="6e89aed05f8f176233aa966974700902f92ccd45d4e7ce72a6d201a0d7e54a27" exitCode=7
Jul 16 22:49:06.464056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:06.463677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" event={"ID":"f905ce42-5e1f-4572-89f1-8206da9486ad","Type":"ContainerDied","Data":"6e89aed05f8f176233aa966974700902f92ccd45d4e7ce72a6d201a0d7e54a27"}
Jul 16 22:49:06.464056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:06.463732    2564 scope.go:117] "RemoveContainer" containerID="6c6152211a48daa2df221399a4117d46356ecd009d342c2c5708d561390f7e24"
Jul 16 22:49:06.464056 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:06.463967    2564 scope.go:117] "RemoveContainer" containerID="6e89aed05f8f176233aa966974700902f92ccd45d4e7ce72a6d201a0d7e54a27"
Jul 16 22:49:06.464235 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:06.464207    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-5cx9t_odh-ai-gateway-infra(f905ce42-5e1f-4572-89f1-8206da9486ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad"
Jul 16 22:49:07.490576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:07.490540    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"]
Jul 16 22:49:07.624768 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:07.624744    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:49:07.740307 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:07.740273    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f905ce42-5e1f-4572-89f1-8206da9486ad-maas-api-service-ca\") pod \"f905ce42-5e1f-4572-89f1-8206da9486ad\" (UID: \"f905ce42-5e1f-4572-89f1-8206da9486ad\") "
Jul 16 22:49:07.740749 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:07.740675    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f905ce42-5e1f-4572-89f1-8206da9486ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f905ce42-5e1f-4572-89f1-8206da9486ad" (UID: "f905ce42-5e1f-4572-89f1-8206da9486ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:49:07.841898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:07.841862    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f905ce42-5e1f-4572-89f1-8206da9486ad-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:49:08.478033 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:08.478006    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"
Jul 16 22:49:08.478033 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:08.478014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t" event={"ID":"f905ce42-5e1f-4572-89f1-8206da9486ad","Type":"ContainerDied","Data":"67de9b154c32ca0ee3e83d16ad36843ea93e943aa98f553ae01cdcbf5174b82d"}
Jul 16 22:49:08.478296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:08.478069    2564 scope.go:117] "RemoveContainer" containerID="6e89aed05f8f176233aa966974700902f92ccd45d4e7ce72a6d201a0d7e54a27"
Jul 16 22:49:08.501626 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:08.501588    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"]
Jul 16 22:49:08.510976 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:08.510945    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5cx9t"]
Jul 16 22:49:09.602466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:09.602433    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" path="/var/lib/kubelet/pods/f905ce42-5e1f-4572-89f1-8206da9486ad/volumes"
Jul 16 22:49:19.652502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652468    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"]
Jul 16 22:49:19.652892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652798    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.652892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652810    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.652892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652825    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.652892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652830    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.653020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652904    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.653020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652912    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.653020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.652918    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:19.657194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.657174    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:19.664121 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.664092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"]
Jul 16 22:49:19.744163 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.744108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn\" (UID: \"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:19.845502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.845465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn\" (UID: \"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:19.846099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.846079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn\" (UID: \"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:19.968760 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:19.968724    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:20.311296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:20.311269    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"]
Jul 16 22:49:20.313502 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:49:20.313472    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d85bec9_a0cb_4efe_9ac7_7bea12b9ea76.slice/crio-3be38fec8abfcd07aa050377a5a24c767b5c598c4ec498deb464c817375cbae2 WatchSource:0}: Error finding container 3be38fec8abfcd07aa050377a5a24c767b5c598c4ec498deb464c817375cbae2: Status 404 returned error can't find the container with id 3be38fec8abfcd07aa050377a5a24c767b5c598c4ec498deb464c817375cbae2
Jul 16 22:49:20.544157 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:20.544102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" event={"ID":"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76","Type":"ContainerStarted","Data":"d17351194518af342f6680c9e116b6e01c747d1534b89918eba03e385d6ac4ec"}
Jul 16 22:49:20.544326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:20.544182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" event={"ID":"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76","Type":"ContainerStarted","Data":"3be38fec8abfcd07aa050377a5a24c767b5c598c4ec498deb464c817375cbae2"}
Jul 16 22:49:20.560859 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:20.560768    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" podStartSLOduration=1.560752897 podStartE2EDuration="1.560752897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:49:20.55885492 +0000 UTC m=+1916.879410993" watchObservedRunningTime="2026-07-16 22:49:20.560752897 +0000 UTC m=+1916.881308968"
Jul 16 22:49:20.596923 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:20.596898    2564 scope.go:117] "RemoveContainer" containerID="94273038198bca0127fa440963af4c104beeb55d21d77adbafb6bf1809db0e72"
Jul 16 22:49:21.551840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.551807    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerID="203a796837bd0eac6f68162a0b9f94164a5ff353ca4da920c68fbe07810d18f9" exitCode=6
Jul 16 22:49:21.552317 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.551879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" event={"ID":"e4951d3b-f0bd-4243-9866-3a862410a7dc","Type":"ContainerDied","Data":"203a796837bd0eac6f68162a0b9f94164a5ff353ca4da920c68fbe07810d18f9"}
Jul 16 22:49:21.552317 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.551935    2564 scope.go:117] "RemoveContainer" containerID="94273038198bca0127fa440963af4c104beeb55d21d77adbafb6bf1809db0e72"
Jul 16 22:49:21.552317 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.552240    2564 scope.go:117] "RemoveContainer" containerID="203a796837bd0eac6f68162a0b9f94164a5ff353ca4da920c68fbe07810d18f9"
Jul 16 22:49:21.552527 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:21.552508    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb_odh-ai-gateway-infra(e4951d3b-f0bd-4243-9866-3a862410a7dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc"
Jul 16 22:49:21.553664 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.553642    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerID="d17351194518af342f6680c9e116b6e01c747d1534b89918eba03e385d6ac4ec" exitCode=6
Jul 16 22:49:21.553770 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.553706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" event={"ID":"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76","Type":"ContainerDied","Data":"d17351194518af342f6680c9e116b6e01c747d1534b89918eba03e385d6ac4ec"}
Jul 16 22:49:21.553851 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:21.553838    2564 scope.go:117] "RemoveContainer" containerID="d17351194518af342f6680c9e116b6e01c747d1534b89918eba03e385d6ac4ec"
Jul 16 22:49:22.561154 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.561051    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerID="809cf9d68d87615e143d937ff6a15c661983f780ac3f048be318be2b49fdbd9b" exitCode=6
Jul 16 22:49:22.561154 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.561118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" event={"ID":"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76","Type":"ContainerDied","Data":"809cf9d68d87615e143d937ff6a15c661983f780ac3f048be318be2b49fdbd9b"}
Jul 16 22:49:22.561657 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.561199    2564 scope.go:117] "RemoveContainer" containerID="d17351194518af342f6680c9e116b6e01c747d1534b89918eba03e385d6ac4ec"
Jul 16 22:49:22.561657 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.561399    2564 scope.go:117] "RemoveContainer" containerID="809cf9d68d87615e143d937ff6a15c661983f780ac3f048be318be2b49fdbd9b"
Jul 16 22:49:22.561657 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:22.561634    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn_odh-ai-gateway-infra(5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76"
Jul 16 22:49:22.581545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.581518    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"]
Jul 16 22:49:22.722825 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.722795    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:22.765914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.765884    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4951d3b-f0bd-4243-9866-3a862410a7dc-maas-api-service-ca\") pod \"e4951d3b-f0bd-4243-9866-3a862410a7dc\" (UID: \"e4951d3b-f0bd-4243-9866-3a862410a7dc\") "
Jul 16 22:49:22.766247 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.766226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4951d3b-f0bd-4243-9866-3a862410a7dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4951d3b-f0bd-4243-9866-3a862410a7dc" (UID: "e4951d3b-f0bd-4243-9866-3a862410a7dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:49:22.867231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:22.867155    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4951d3b-f0bd-4243-9866-3a862410a7dc-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:49:23.570192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:23.570165    2564 scope.go:117] "RemoveContainer" containerID="809cf9d68d87615e143d937ff6a15c661983f780ac3f048be318be2b49fdbd9b"
Jul 16 22:49:23.570666 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:23.570431    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn_odh-ai-gateway-infra(5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76"
Jul 16 22:49:23.571159 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:23.571121    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"
Jul 16 22:49:23.571243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:23.571170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb" event={"ID":"e4951d3b-f0bd-4243-9866-3a862410a7dc","Type":"ContainerDied","Data":"e2cb330bd09208ae802d54a24ef41e9ad7d659374b24da42ae8c6bd57b852812"}
Jul 16 22:49:23.571243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:23.571234    2564 scope.go:117] "RemoveContainer" containerID="203a796837bd0eac6f68162a0b9f94164a5ff353ca4da920c68fbe07810d18f9"
Jul 16 22:49:23.602064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:23.602038    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"]
Jul 16 22:49:23.607097 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:23.607072    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwqsb"]
Jul 16 22:49:25.601784 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:25.601751    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" path="/var/lib/kubelet/pods/e4951d3b-f0bd-4243-9866-3a862410a7dc/volumes"
Jul 16 22:49:37.596411 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:37.596375    2564 scope.go:117] "RemoveContainer" containerID="809cf9d68d87615e143d937ff6a15c661983f780ac3f048be318be2b49fdbd9b"
Jul 16 22:49:38.649624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:38.649590    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerID="9e71e4e002099b0a660f90949fc85f015501b0f683ef94cd1c6905aed3c65eb4" exitCode=6
Jul 16 22:49:38.650108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:38.649652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" event={"ID":"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76","Type":"ContainerDied","Data":"9e71e4e002099b0a660f90949fc85f015501b0f683ef94cd1c6905aed3c65eb4"}
Jul 16 22:49:38.650108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:38.649711    2564 scope.go:117] "RemoveContainer" containerID="809cf9d68d87615e143d937ff6a15c661983f780ac3f048be318be2b49fdbd9b"
Jul 16 22:49:38.650108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:38.649987    2564 scope.go:117] "RemoveContainer" containerID="9e71e4e002099b0a660f90949fc85f015501b0f683ef94cd1c6905aed3c65eb4"
Jul 16 22:49:38.650279 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:38.650250    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn_odh-ai-gateway-infra(5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76"
Jul 16 22:49:39.679108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:39.679070    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"]
Jul 16 22:49:39.812609 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:39.812586    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:39.910227 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:39.910190    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76-maas-api-service-ca\") pod \"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76\" (UID: \"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76\") "
Jul 16 22:49:39.910524 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:39.910501    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" (UID: "5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:49:40.011715 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:40.011679    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:49:40.662333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:40.662300    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"
Jul 16 22:49:40.662530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:40.662348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn" event={"ID":"5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76","Type":"ContainerDied","Data":"3be38fec8abfcd07aa050377a5a24c767b5c598c4ec498deb464c817375cbae2"}
Jul 16 22:49:40.662530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:40.662406    2564 scope.go:117] "RemoveContainer" containerID="9e71e4e002099b0a660f90949fc85f015501b0f683ef94cd1c6905aed3c65eb4"
Jul 16 22:49:40.683821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:40.683795    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"]
Jul 16 22:49:40.692943 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:40.692919    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-hhxcn"]
Jul 16 22:49:41.601597 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:41.601561    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" path="/var/lib/kubelet/pods/5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76/volumes"
Jul 16 22:49:48.244333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244292    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"]
Jul 16 22:49:48.244806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244764    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:49:48.244806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244785    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:49:48.244806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244796    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:49:48.244806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244805    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244825    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244834    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244845    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244853    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f905ce42-5e1f-4572-89f1-8206da9486ad" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244874    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244882    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244972    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244984    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.244995    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.245004    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.245029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.245015    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:49:48.249442 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.249421    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:49:48.252259 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.252238    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:49:48.252730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.252713    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:49:48.257104 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.257072    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"]
Jul 16 22:49:48.390351 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.390316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c9677b-f5bf-4843-b525-b0be8c9f5f78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-g4cd5\" (UID: \"c1c9677b-f5bf-4843-b525-b0be8c9f5f78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:49:48.491591 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.491553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c9677b-f5bf-4843-b525-b0be8c9f5f78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-g4cd5\" (UID: \"c1c9677b-f5bf-4843-b525-b0be8c9f5f78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:49:48.492203 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.492183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c9677b-f5bf-4843-b525-b0be8c9f5f78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-g4cd5\" (UID: \"c1c9677b-f5bf-4843-b525-b0be8c9f5f78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:49:48.560862 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.560789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:49:48.697519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.697489    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"]
Jul 16 22:49:48.699314 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:49:48.699284    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c9677b_f5bf_4843_b525_b0be8c9f5f78.slice/crio-73a4a276415cd1b4ccfc8c72d686b32a9504ff8c5cda83c8099d2945863b00e1 WatchSource:0}: Error finding container 73a4a276415cd1b4ccfc8c72d686b32a9504ff8c5cda83c8099d2945863b00e1: Status 404 returned error can't find the container with id 73a4a276415cd1b4ccfc8c72d686b32a9504ff8c5cda83c8099d2945863b00e1
Jul 16 22:49:48.707940 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:48.707916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" event={"ID":"c1c9677b-f5bf-4843-b525-b0be8c9f5f78","Type":"ContainerStarted","Data":"73a4a276415cd1b4ccfc8c72d686b32a9504ff8c5cda83c8099d2945863b00e1"}
Jul 16 22:49:49.714221 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:49.714184    2564 generic.go:356] "Generic (PLEG): container finished" podID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerID="4b6ae0d3841a384e1c625d47bed1c6bb02d9929600eb7437ec7fb9d42228e16a" exitCode=7
Jul 16 22:49:49.714613 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:49.714260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" event={"ID":"c1c9677b-f5bf-4843-b525-b0be8c9f5f78","Type":"ContainerDied","Data":"4b6ae0d3841a384e1c625d47bed1c6bb02d9929600eb7437ec7fb9d42228e16a"}
Jul 16 22:49:49.714613 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:49.714461    2564 scope.go:117] "RemoveContainer" containerID="4b6ae0d3841a384e1c625d47bed1c6bb02d9929600eb7437ec7fb9d42228e16a"
Jul 16 22:49:50.721153 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:50.721106    2564 generic.go:356] "Generic (PLEG): container finished" podID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerID="628f8d15b13e4bf92c32e491b46104322844465ba36fa6594dbbbe1f7e78984a" exitCode=7
Jul 16 22:49:50.721601 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:50.721197    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" event={"ID":"c1c9677b-f5bf-4843-b525-b0be8c9f5f78","Type":"ContainerDied","Data":"628f8d15b13e4bf92c32e491b46104322844465ba36fa6594dbbbe1f7e78984a"}
Jul 16 22:49:50.721601 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:50.721263    2564 scope.go:117] "RemoveContainer" containerID="4b6ae0d3841a384e1c625d47bed1c6bb02d9929600eb7437ec7fb9d42228e16a"
Jul 16 22:49:50.721601 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:50.721440    2564 scope.go:117] "RemoveContainer" containerID="628f8d15b13e4bf92c32e491b46104322844465ba36fa6594dbbbe1f7e78984a"
Jul 16 22:49:50.721736 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:50.721688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-g4cd5_odh-ai-gateway-infra(c1c9677b-f5bf-4843-b525-b0be8c9f5f78)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78"
Jul 16 22:49:51.728481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:49:51.728453    2564 scope.go:117] "RemoveContainer" containerID="628f8d15b13e4bf92c32e491b46104322844465ba36fa6594dbbbe1f7e78984a"
Jul 16 22:49:51.728855 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:49:51.728629    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-g4cd5_odh-ai-gateway-infra(c1c9677b-f5bf-4843-b525-b0be8c9f5f78)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78"
Jul 16 22:50:01.632594 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.632561    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"]
Jul 16 22:50:01.632978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.632891    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:50:01.632978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.632901    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:50:01.632978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.632914    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:50:01.632978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.632919    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d85bec9-a0cb-4efe-9ac7-7bea12b9ea76" containerName="revoke-keys"
Jul 16 22:50:01.632978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.632980    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4951d3b-f0bd-4243-9866-3a862410a7dc" containerName="revoke-keys"
Jul 16 22:50:01.635920 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.635903    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:01.643340 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.643298    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"]
Jul 16 22:50:01.708606 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.708570    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv\" (UID: \"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:01.809428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.809396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv\" (UID: \"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:01.810039 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.810017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv\" (UID: \"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:01.947749 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:01.947719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:02.080201 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:02.080175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"]
Jul 16 22:50:02.082039 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:50:02.082014    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb86e2f_2152_443c_a9ff_a46fd9c4e63f.slice/crio-df6460b2b776e95d5bf350b5fe5ba2b6845c7b71b35b81c2fb61ec1cca6fae40 WatchSource:0}: Error finding container df6460b2b776e95d5bf350b5fe5ba2b6845c7b71b35b81c2fb61ec1cca6fae40: Status 404 returned error can't find the container with id df6460b2b776e95d5bf350b5fe5ba2b6845c7b71b35b81c2fb61ec1cca6fae40
Jul 16 22:50:02.783812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:02.783786    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerID="e486c6577671d8a4fcdd75546f4c5cb90ce238909ff3c9e04e129f9ebef1f0b8" exitCode=6
Jul 16 22:50:02.784119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:02.783863    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" event={"ID":"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f","Type":"ContainerDied","Data":"e486c6577671d8a4fcdd75546f4c5cb90ce238909ff3c9e04e129f9ebef1f0b8"}
Jul 16 22:50:02.784119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:02.783909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" event={"ID":"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f","Type":"ContainerStarted","Data":"df6460b2b776e95d5bf350b5fe5ba2b6845c7b71b35b81c2fb61ec1cca6fae40"}
Jul 16 22:50:02.784119 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:02.784033    2564 scope.go:117] "RemoveContainer" containerID="e486c6577671d8a4fcdd75546f4c5cb90ce238909ff3c9e04e129f9ebef1f0b8"
Jul 16 22:50:03.790993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:03.790956    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerID="65e477c874b1e786dfc9f2e99ceb9aabf0f2628ad195e353b31032f27db85044" exitCode=6
Jul 16 22:50:03.791433 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:03.791043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" event={"ID":"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f","Type":"ContainerDied","Data":"65e477c874b1e786dfc9f2e99ceb9aabf0f2628ad195e353b31032f27db85044"}
Jul 16 22:50:03.791433 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:03.791101    2564 scope.go:117] "RemoveContainer" containerID="e486c6577671d8a4fcdd75546f4c5cb90ce238909ff3c9e04e129f9ebef1f0b8"
Jul 16 22:50:03.791433 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:03.791233    2564 scope.go:117] "RemoveContainer" containerID="65e477c874b1e786dfc9f2e99ceb9aabf0f2628ad195e353b31032f27db85044"
Jul 16 22:50:03.791543 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:03.791511    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv_odh-ai-gateway-infra(ffb86e2f-2152-443c-a9ff-a46fd9c4e63f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f"
Jul 16 22:50:04.798013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:04.797984    2564 scope.go:117] "RemoveContainer" containerID="65e477c874b1e786dfc9f2e99ceb9aabf0f2628ad195e353b31032f27db85044"
Jul 16 22:50:04.798438 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:04.798197    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv_odh-ai-gateway-infra(ffb86e2f-2152-443c-a9ff-a46fd9c4e63f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f"
Jul 16 22:50:05.600156 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:05.600100    2564 scope.go:117] "RemoveContainer" containerID="628f8d15b13e4bf92c32e491b46104322844465ba36fa6594dbbbe1f7e78984a"
Jul 16 22:50:06.810404 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:06.810363    2564 generic.go:356] "Generic (PLEG): container finished" podID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerID="6d344190a53698f6b561e4eef840a56b52294500b1363d496454a5e220b9370f" exitCode=7
Jul 16 22:50:06.810899 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:06.810441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" event={"ID":"c1c9677b-f5bf-4843-b525-b0be8c9f5f78","Type":"ContainerDied","Data":"6d344190a53698f6b561e4eef840a56b52294500b1363d496454a5e220b9370f"}
Jul 16 22:50:06.810899 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:06.810517    2564 scope.go:117] "RemoveContainer" containerID="628f8d15b13e4bf92c32e491b46104322844465ba36fa6594dbbbe1f7e78984a"
Jul 16 22:50:06.810899 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:06.810863    2564 scope.go:117] "RemoveContainer" containerID="6d344190a53698f6b561e4eef840a56b52294500b1363d496454a5e220b9370f"
Jul 16 22:50:06.811103 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:06.811081    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-g4cd5_odh-ai-gateway-infra(c1c9677b-f5bf-4843-b525-b0be8c9f5f78)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78"
Jul 16 22:50:07.838118 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:07.838080    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"]
Jul 16 22:50:07.973830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:07.973806    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:50:08.070260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.070224    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c9677b-f5bf-4843-b525-b0be8c9f5f78-maas-api-service-ca\") pod \"c1c9677b-f5bf-4843-b525-b0be8c9f5f78\" (UID: \"c1c9677b-f5bf-4843-b525-b0be8c9f5f78\") "
Jul 16 22:50:08.070591 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.070568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1c9677b-f5bf-4843-b525-b0be8c9f5f78-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1c9677b-f5bf-4843-b525-b0be8c9f5f78" (UID: "c1c9677b-f5bf-4843-b525-b0be8c9f5f78"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:50:08.171273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.171193    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1c9677b-f5bf-4843-b525-b0be8c9f5f78-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:50:08.823928 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.823888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5" event={"ID":"c1c9677b-f5bf-4843-b525-b0be8c9f5f78","Type":"ContainerDied","Data":"73a4a276415cd1b4ccfc8c72d686b32a9504ff8c5cda83c8099d2945863b00e1"}
Jul 16 22:50:08.823928 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.823914    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"
Jul 16 22:50:08.824329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.823948    2564 scope.go:117] "RemoveContainer" containerID="6d344190a53698f6b561e4eef840a56b52294500b1363d496454a5e220b9370f"
Jul 16 22:50:08.851073 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.847589    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"]
Jul 16 22:50:08.854484 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:08.854459    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g4cd5"]
Jul 16 22:50:09.602218 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:09.602177    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" path="/var/lib/kubelet/pods/c1c9677b-f5bf-4843-b525-b0be8c9f5f78/volumes"
Jul 16 22:50:16.596202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:16.596111    2564 scope.go:117] "RemoveContainer" containerID="65e477c874b1e786dfc9f2e99ceb9aabf0f2628ad195e353b31032f27db85044"
Jul 16 22:50:16.867429 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:16.867330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" event={"ID":"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f","Type":"ContainerStarted","Data":"3dc7b14198b1f41c7d41a12e1eb3bf2a853ea865dc401bc265ec374abd31b6c8"}
Jul 16 22:50:16.908543 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:16.908486    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" podStartSLOduration=15.90846966 podStartE2EDuration="15.90846966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:50:16.907848175 +0000 UTC m=+1973.228404249" watchObservedRunningTime="2026-07-16 22:50:16.90846966 +0000 UTC m=+1973.229025732"
Jul 16 22:50:17.874344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:17.874311    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerID="3dc7b14198b1f41c7d41a12e1eb3bf2a853ea865dc401bc265ec374abd31b6c8" exitCode=6
Jul 16 22:50:17.874835 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:17.874371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" event={"ID":"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f","Type":"ContainerDied","Data":"3dc7b14198b1f41c7d41a12e1eb3bf2a853ea865dc401bc265ec374abd31b6c8"}
Jul 16 22:50:17.874835 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:17.874431    2564 scope.go:117] "RemoveContainer" containerID="65e477c874b1e786dfc9f2e99ceb9aabf0f2628ad195e353b31032f27db85044"
Jul 16 22:50:17.874835 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:17.874668    2564 scope.go:117] "RemoveContainer" containerID="3dc7b14198b1f41c7d41a12e1eb3bf2a853ea865dc401bc265ec374abd31b6c8"
Jul 16 22:50:17.874987 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:17.874891    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv_odh-ai-gateway-infra(ffb86e2f-2152-443c-a9ff-a46fd9c4e63f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f"
Jul 16 22:50:17.917178 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:17.917127    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"]
Jul 16 22:50:19.014265 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.014240    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:19.072782 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.072747    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f-maas-api-service-ca\") pod \"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f\" (UID: \"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f\") "
Jul 16 22:50:19.073176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.073129    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" (UID: "ffb86e2f-2152-443c-a9ff-a46fd9c4e63f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:50:19.174431 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.174342    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:50:19.712702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.712663    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"]
Jul 16 22:50:19.713045 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713024    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713047    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713054    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713059    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713066    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713072    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713081    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:19.713090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713086    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:19.713344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713161    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:19.713344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713170    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:19.713344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713176    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:19.713344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.713182    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:19.716324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.716309    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:19.726326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.726296    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"]
Jul 16 22:50:19.780781 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.780737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21092d2-145f-4a51-b052-ef8fb7d5376d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6\" (UID: \"d21092d2-145f-4a51-b052-ef8fb7d5376d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:19.881804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.881716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21092d2-145f-4a51-b052-ef8fb7d5376d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6\" (UID: \"d21092d2-145f-4a51-b052-ef8fb7d5376d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:19.882407 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.882388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21092d2-145f-4a51-b052-ef8fb7d5376d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6\" (UID: \"d21092d2-145f-4a51-b052-ef8fb7d5376d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:19.887764 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.887743    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"
Jul 16 22:50:19.887912 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.887768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv" event={"ID":"ffb86e2f-2152-443c-a9ff-a46fd9c4e63f","Type":"ContainerDied","Data":"df6460b2b776e95d5bf350b5fe5ba2b6845c7b71b35b81c2fb61ec1cca6fae40"}
Jul 16 22:50:19.887912 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.887836    2564 scope.go:117] "RemoveContainer" containerID="3dc7b14198b1f41c7d41a12e1eb3bf2a853ea865dc401bc265ec374abd31b6c8"
Jul 16 22:50:19.910405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.910377    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"]
Jul 16 22:50:19.913452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:19.913426    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-vcdrv"]
Jul 16 22:50:20.028051 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:20.028016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:20.166991 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:20.166962    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"]
Jul 16 22:50:20.168149 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:50:20.168106    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21092d2_145f_4a51_b052_ef8fb7d5376d.slice/crio-dd435ab24f3003e7fcc08ac24c9ff5a82e9980f2838be35a8175d501fb98c321 WatchSource:0}: Error finding container dd435ab24f3003e7fcc08ac24c9ff5a82e9980f2838be35a8175d501fb98c321: Status 404 returned error can't find the container with id dd435ab24f3003e7fcc08ac24c9ff5a82e9980f2838be35a8175d501fb98c321
Jul 16 22:50:20.896032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:20.896000    2564 generic.go:356] "Generic (PLEG): container finished" podID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerID="b4c3ed7c0ae5674250ceb11b3e9fda10b5a446306ba80d2f097febf231327148" exitCode=6
Jul 16 22:50:20.896032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:20.896032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" event={"ID":"d21092d2-145f-4a51-b052-ef8fb7d5376d","Type":"ContainerDied","Data":"b4c3ed7c0ae5674250ceb11b3e9fda10b5a446306ba80d2f097febf231327148"}
Jul 16 22:50:20.896271 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:20.896067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" event={"ID":"d21092d2-145f-4a51-b052-ef8fb7d5376d","Type":"ContainerStarted","Data":"dd435ab24f3003e7fcc08ac24c9ff5a82e9980f2838be35a8175d501fb98c321"}
Jul 16 22:50:20.896358 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:20.896302    2564 scope.go:117] "RemoveContainer" containerID="b4c3ed7c0ae5674250ceb11b3e9fda10b5a446306ba80d2f097febf231327148"
Jul 16 22:50:21.602447 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:21.602414    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" path="/var/lib/kubelet/pods/ffb86e2f-2152-443c-a9ff-a46fd9c4e63f/volumes"
Jul 16 22:50:21.903616 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:21.903531    2564 generic.go:356] "Generic (PLEG): container finished" podID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerID="f2bc09ff331fe16a1249f8ceb8dc8690d5fc9267173dc91445dcf609c79092bc" exitCode=6
Jul 16 22:50:21.903761 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:21.903617    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" event={"ID":"d21092d2-145f-4a51-b052-ef8fb7d5376d","Type":"ContainerDied","Data":"f2bc09ff331fe16a1249f8ceb8dc8690d5fc9267173dc91445dcf609c79092bc"}
Jul 16 22:50:21.903761 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:21.903681    2564 scope.go:117] "RemoveContainer" containerID="b4c3ed7c0ae5674250ceb11b3e9fda10b5a446306ba80d2f097febf231327148"
Jul 16 22:50:21.903875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:21.903849    2564 scope.go:117] "RemoveContainer" containerID="f2bc09ff331fe16a1249f8ceb8dc8690d5fc9267173dc91445dcf609c79092bc"
Jul 16 22:50:21.904081 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:21.904058    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6_odh-ai-gateway-infra(d21092d2-145f-4a51-b052-ef8fb7d5376d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d"
Jul 16 22:50:22.911692 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:22.911665    2564 scope.go:117] "RemoveContainer" containerID="f2bc09ff331fe16a1249f8ceb8dc8690d5fc9267173dc91445dcf609c79092bc"
Jul 16 22:50:22.912060 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:22.911837    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6_odh-ai-gateway-infra(d21092d2-145f-4a51-b052-ef8fb7d5376d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d"
Jul 16 22:50:37.596047 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:37.596014    2564 scope.go:117] "RemoveContainer" containerID="f2bc09ff331fe16a1249f8ceb8dc8690d5fc9267173dc91445dcf609c79092bc"
Jul 16 22:50:37.998807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:37.998755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" event={"ID":"d21092d2-145f-4a51-b052-ef8fb7d5376d","Type":"ContainerStarted","Data":"3d1dcae8da60e535e06e3cdbb591d9db8201c7e7ac8e2978a600a5ab9e088bda"}
Jul 16 22:50:38.018999 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:38.018948    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" podStartSLOduration=19.018930498 podStartE2EDuration="19.018930498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:50:38.01738121 +0000 UTC m=+1994.337937282" watchObservedRunningTime="2026-07-16 22:50:38.018930498 +0000 UTC m=+1994.339486572"
Jul 16 22:50:39.005828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:39.005797    2564 generic.go:356] "Generic (PLEG): container finished" podID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerID="3d1dcae8da60e535e06e3cdbb591d9db8201c7e7ac8e2978a600a5ab9e088bda" exitCode=6
Jul 16 22:50:39.006347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:39.005874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" event={"ID":"d21092d2-145f-4a51-b052-ef8fb7d5376d","Type":"ContainerDied","Data":"3d1dcae8da60e535e06e3cdbb591d9db8201c7e7ac8e2978a600a5ab9e088bda"}
Jul 16 22:50:39.006347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:39.005956    2564 scope.go:117] "RemoveContainer" containerID="f2bc09ff331fe16a1249f8ceb8dc8690d5fc9267173dc91445dcf609c79092bc"
Jul 16 22:50:39.006347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:39.006268    2564 scope.go:117] "RemoveContainer" containerID="3d1dcae8da60e535e06e3cdbb591d9db8201c7e7ac8e2978a600a5ab9e088bda"
Jul 16 22:50:39.006502 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:39.006482    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6_odh-ai-gateway-infra(d21092d2-145f-4a51-b052-ef8fb7d5376d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d"
Jul 16 22:50:39.034472 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:39.034445    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"]
Jul 16 22:50:40.142993 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:40.142964    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:40.256900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:40.256861    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21092d2-145f-4a51-b052-ef8fb7d5376d-maas-api-service-ca\") pod \"d21092d2-145f-4a51-b052-ef8fb7d5376d\" (UID: \"d21092d2-145f-4a51-b052-ef8fb7d5376d\") "
Jul 16 22:50:40.257249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:40.257225    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21092d2-145f-4a51-b052-ef8fb7d5376d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d21092d2-145f-4a51-b052-ef8fb7d5376d" (UID: "d21092d2-145f-4a51-b052-ef8fb7d5376d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:50:40.358694 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:40.358603    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21092d2-145f-4a51-b052-ef8fb7d5376d-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:50:41.019342 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:41.019317    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"
Jul 16 22:50:41.019526 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:41.019356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6" event={"ID":"d21092d2-145f-4a51-b052-ef8fb7d5376d","Type":"ContainerDied","Data":"dd435ab24f3003e7fcc08ac24c9ff5a82e9980f2838be35a8175d501fb98c321"}
Jul 16 22:50:41.019526 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:41.019421    2564 scope.go:117] "RemoveContainer" containerID="3d1dcae8da60e535e06e3cdbb591d9db8201c7e7ac8e2978a600a5ab9e088bda"
Jul 16 22:50:41.040484 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:41.040457    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"]
Jul 16 22:50:41.049944 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:41.049917    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-bf4v6"]
Jul 16 22:50:41.601773 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:41.601741    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" path="/var/lib/kubelet/pods/d21092d2-145f-4a51-b052-ef8fb7d5376d/volumes"
Jul 16 22:50:48.294031 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.293997    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"]
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294343    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294355    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294367    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294373    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294383    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294389    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294401    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294407    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294414    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294438 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294419    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294470    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.294747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294479    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1c9677b-f5bf-4843-b525-b0be8c9f5f78" containerName="revoke-keys"
Jul 16 22:50:48.294747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294487    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb86e2f-2152-443c-a9ff-a46fd9c4e63f" containerName="revoke-keys"
Jul 16 22:50:48.294747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.294493    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:48.298959 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.298942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:50:48.301695 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.301672    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:50:48.301695 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.301689    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:50:48.306512 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.306469    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"]
Jul 16 22:50:48.430499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.430462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eac7f40a-054c-4f60-81de-29ae18e282c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-qpj5x\" (UID: \"eac7f40a-054c-4f60-81de-29ae18e282c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:50:48.531079 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.531044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eac7f40a-054c-4f60-81de-29ae18e282c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-qpj5x\" (UID: \"eac7f40a-054c-4f60-81de-29ae18e282c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:50:48.531693 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.531674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eac7f40a-054c-4f60-81de-29ae18e282c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-qpj5x\" (UID: \"eac7f40a-054c-4f60-81de-29ae18e282c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:50:48.611812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.611738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:50:48.752953 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:48.752926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"]
Jul 16 22:50:48.754325 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:50:48.754301    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac7f40a_054c_4f60_81de_29ae18e282c5.slice/crio-7fbd82d9b6857d7bbaf6349c566052e8ce0528d3ea6c7466ddbd875daff2776f WatchSource:0}: Error finding container 7fbd82d9b6857d7bbaf6349c566052e8ce0528d3ea6c7466ddbd875daff2776f: Status 404 returned error can't find the container with id 7fbd82d9b6857d7bbaf6349c566052e8ce0528d3ea6c7466ddbd875daff2776f
Jul 16 22:50:49.061700 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:49.061665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" event={"ID":"eac7f40a-054c-4f60-81de-29ae18e282c5","Type":"ContainerStarted","Data":"7fbd82d9b6857d7bbaf6349c566052e8ce0528d3ea6c7466ddbd875daff2776f"}
Jul 16 22:50:50.068410 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:50.068371    2564 generic.go:356] "Generic (PLEG): container finished" podID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerID="2fab51cabb494ee0898d678681c7bda8b35ec90ac392e994e68e5929231ac3ee" exitCode=7
Jul 16 22:50:50.068799 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:50.068454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" event={"ID":"eac7f40a-054c-4f60-81de-29ae18e282c5","Type":"ContainerDied","Data":"2fab51cabb494ee0898d678681c7bda8b35ec90ac392e994e68e5929231ac3ee"}
Jul 16 22:50:50.068799 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:50.068598    2564 scope.go:117] "RemoveContainer" containerID="2fab51cabb494ee0898d678681c7bda8b35ec90ac392e994e68e5929231ac3ee"
Jul 16 22:50:51.075452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.075417    2564 generic.go:356] "Generic (PLEG): container finished" podID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerID="91d5139409fda64186c6ac865df41e4c895871d4112455a374d657c20d9baeae" exitCode=7
Jul 16 22:50:51.075908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.075480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" event={"ID":"eac7f40a-054c-4f60-81de-29ae18e282c5","Type":"ContainerDied","Data":"91d5139409fda64186c6ac865df41e4c895871d4112455a374d657c20d9baeae"}
Jul 16 22:50:51.075908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.075540    2564 scope.go:117] "RemoveContainer" containerID="2fab51cabb494ee0898d678681c7bda8b35ec90ac392e994e68e5929231ac3ee"
Jul 16 22:50:51.075908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.075742    2564 scope.go:117] "RemoveContainer" containerID="91d5139409fda64186c6ac865df41e4c895871d4112455a374d657c20d9baeae"
Jul 16 22:50:51.076083 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:51.075990    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-qpj5x_odh-ai-gateway-infra(eac7f40a-054c-4f60-81de-29ae18e282c5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5"
Jul 16 22:50:51.690700 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.690667    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"]
Jul 16 22:50:51.691071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.691057    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21092d2-145f-4a51-b052-ef8fb7d5376d" containerName="revoke-keys"
Jul 16 22:50:51.695108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.695091    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:50:51.705499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.705468    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"]
Jul 16 22:50:51.861165 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.861101    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1038b3-1f3a-4306-a464-a80a5963fe03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx\" (UID: \"9b1038b3-1f3a-4306-a464-a80a5963fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:50:51.962292 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.962259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1038b3-1f3a-4306-a464-a80a5963fe03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx\" (UID: \"9b1038b3-1f3a-4306-a464-a80a5963fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:50:51.962961 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:51.962933    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1038b3-1f3a-4306-a464-a80a5963fe03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx\" (UID: \"9b1038b3-1f3a-4306-a464-a80a5963fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:50:52.008477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:52.008444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:50:52.083069 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:52.083045    2564 scope.go:117] "RemoveContainer" containerID="91d5139409fda64186c6ac865df41e4c895871d4112455a374d657c20d9baeae"
Jul 16 22:50:52.083454 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:52.083267    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-qpj5x_odh-ai-gateway-infra(eac7f40a-054c-4f60-81de-29ae18e282c5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5"
Jul 16 22:50:52.350416 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:52.350382    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"]
Jul 16 22:50:52.351489 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:50:52.351466    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1038b3_1f3a_4306_a464_a80a5963fe03.slice/crio-0a67e974348c6e223110cd814a1fc80eb1f30a256d0007558e6fa05296c2e442 WatchSource:0}: Error finding container 0a67e974348c6e223110cd814a1fc80eb1f30a256d0007558e6fa05296c2e442: Status 404 returned error can't find the container with id 0a67e974348c6e223110cd814a1fc80eb1f30a256d0007558e6fa05296c2e442
Jul 16 22:50:53.090046 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:53.090010    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerID="f38dea24e20fcff61ef2a5785e1fa1cad06fce58ac2584104632a3dc931a2f0f" exitCode=6
Jul 16 22:50:53.090454 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:53.090103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" event={"ID":"9b1038b3-1f3a-4306-a464-a80a5963fe03","Type":"ContainerDied","Data":"f38dea24e20fcff61ef2a5785e1fa1cad06fce58ac2584104632a3dc931a2f0f"}
Jul 16 22:50:53.090454 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:53.090169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" event={"ID":"9b1038b3-1f3a-4306-a464-a80a5963fe03","Type":"ContainerStarted","Data":"0a67e974348c6e223110cd814a1fc80eb1f30a256d0007558e6fa05296c2e442"}
Jul 16 22:50:53.090454 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:53.090296    2564 scope.go:117] "RemoveContainer" containerID="f38dea24e20fcff61ef2a5785e1fa1cad06fce58ac2584104632a3dc931a2f0f"
Jul 16 22:50:54.097404 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:54.097373    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerID="ae8ad07e410ed479e3af93a864dbaa817ac8fd081727b7215604a8278d9aff26" exitCode=6
Jul 16 22:50:54.097808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:54.097460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" event={"ID":"9b1038b3-1f3a-4306-a464-a80a5963fe03","Type":"ContainerDied","Data":"ae8ad07e410ed479e3af93a864dbaa817ac8fd081727b7215604a8278d9aff26"}
Jul 16 22:50:54.097808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:54.097515    2564 scope.go:117] "RemoveContainer" containerID="f38dea24e20fcff61ef2a5785e1fa1cad06fce58ac2584104632a3dc931a2f0f"
Jul 16 22:50:54.097808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:54.097771    2564 scope.go:117] "RemoveContainer" containerID="ae8ad07e410ed479e3af93a864dbaa817ac8fd081727b7215604a8278d9aff26"
Jul 16 22:50:54.097987 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:54.097969    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx_odh-ai-gateway-infra(9b1038b3-1f3a-4306-a464-a80a5963fe03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03"
Jul 16 22:50:55.104734 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:50:55.104704    2564 scope.go:117] "RemoveContainer" containerID="ae8ad07e410ed479e3af93a864dbaa817ac8fd081727b7215604a8278d9aff26"
Jul 16 22:50:55.105093 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:50:55.104905    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx_odh-ai-gateway-infra(9b1038b3-1f3a-4306-a464-a80a5963fe03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03"
Jul 16 22:51:07.595948 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:07.595913    2564 scope.go:117] "RemoveContainer" containerID="91d5139409fda64186c6ac865df41e4c895871d4112455a374d657c20d9baeae"
Jul 16 22:51:08.173809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:08.173733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" event={"ID":"eac7f40a-054c-4f60-81de-29ae18e282c5","Type":"ContainerStarted","Data":"14807f681a12df517191984db408faa32f2f66ce6968bc46a6071726182b2d99"}
Jul 16 22:51:08.191741 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:08.191685    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" podStartSLOduration=20.191669926 podStartE2EDuration="20.191669926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:50:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:51:08.190850094 +0000 UTC m=+2024.511406166" watchObservedRunningTime="2026-07-16 22:51:08.191669926 +0000 UTC m=+2024.512225998"
Jul 16 22:51:08.596240 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:08.596209    2564 scope.go:117] "RemoveContainer" containerID="ae8ad07e410ed479e3af93a864dbaa817ac8fd081727b7215604a8278d9aff26"
Jul 16 22:51:09.182108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.182068    2564 generic.go:356] "Generic (PLEG): container finished" podID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerID="14807f681a12df517191984db408faa32f2f66ce6968bc46a6071726182b2d99" exitCode=7
Jul 16 22:51:09.182402 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.182151    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" event={"ID":"eac7f40a-054c-4f60-81de-29ae18e282c5","Type":"ContainerDied","Data":"14807f681a12df517191984db408faa32f2f66ce6968bc46a6071726182b2d99"}
Jul 16 22:51:09.182402 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.182228    2564 scope.go:117] "RemoveContainer" containerID="91d5139409fda64186c6ac865df41e4c895871d4112455a374d657c20d9baeae"
Jul 16 22:51:09.182620 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.182595    2564 scope.go:117] "RemoveContainer" containerID="14807f681a12df517191984db408faa32f2f66ce6968bc46a6071726182b2d99"
Jul 16 22:51:09.182890 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:09.182866    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-qpj5x_odh-ai-gateway-infra(eac7f40a-054c-4f60-81de-29ae18e282c5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5"
Jul 16 22:51:09.184517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.184497    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" event={"ID":"9b1038b3-1f3a-4306-a464-a80a5963fe03","Type":"ContainerStarted","Data":"a34c8fb52e6a1f3c3418d0dda597fb74857cded77c1af4fef7111bb14e9949e4"}
Jul 16 22:51:09.184788 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.184775    2564 scope.go:117] "RemoveContainer" containerID="a34c8fb52e6a1f3c3418d0dda597fb74857cded77c1af4fef7111bb14e9949e4"
Jul 16 22:51:09.184949 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:09.184933    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx_odh-ai-gateway-infra(9b1038b3-1f3a-4306-a464-a80a5963fe03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03"
Jul 16 22:51:09.214541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:09.214509    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"]
Jul 16 22:51:10.190981 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.190949    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerID="a34c8fb52e6a1f3c3418d0dda597fb74857cded77c1af4fef7111bb14e9949e4" exitCode=6
Jul 16 22:51:10.191467 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.191019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" event={"ID":"9b1038b3-1f3a-4306-a464-a80a5963fe03","Type":"ContainerDied","Data":"a34c8fb52e6a1f3c3418d0dda597fb74857cded77c1af4fef7111bb14e9949e4"}
Jul 16 22:51:10.191467 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.191079    2564 scope.go:117] "RemoveContainer" containerID="ae8ad07e410ed479e3af93a864dbaa817ac8fd081727b7215604a8278d9aff26"
Jul 16 22:51:10.191467 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.191435    2564 scope.go:117] "RemoveContainer" containerID="a34c8fb52e6a1f3c3418d0dda597fb74857cded77c1af4fef7111bb14e9949e4"
Jul 16 22:51:10.191708 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:10.191687    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx_odh-ai-gateway-infra(9b1038b3-1f3a-4306-a464-a80a5963fe03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03"
Jul 16 22:51:10.238064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.238028    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"]
Jul 16 22:51:10.322257 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.322226    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:51:10.418144 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.418113    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eac7f40a-054c-4f60-81de-29ae18e282c5-maas-api-service-ca\") pod \"eac7f40a-054c-4f60-81de-29ae18e282c5\" (UID: \"eac7f40a-054c-4f60-81de-29ae18e282c5\") "
Jul 16 22:51:10.418510 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.418473    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eac7f40a-054c-4f60-81de-29ae18e282c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eac7f40a-054c-4f60-81de-29ae18e282c5" (UID: "eac7f40a-054c-4f60-81de-29ae18e282c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:51:10.519116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:10.519082    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eac7f40a-054c-4f60-81de-29ae18e282c5-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:51:11.200360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.200330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x" event={"ID":"eac7f40a-054c-4f60-81de-29ae18e282c5","Type":"ContainerDied","Data":"7fbd82d9b6857d7bbaf6349c566052e8ce0528d3ea6c7466ddbd875daff2776f"}
Jul 16 22:51:11.200760 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.200393    2564 scope.go:117] "RemoveContainer" containerID="14807f681a12df517191984db408faa32f2f66ce6968bc46a6071726182b2d99"
Jul 16 22:51:11.200760 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.200336    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"
Jul 16 22:51:11.231260 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.231217    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"]
Jul 16 22:51:11.236417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.236385    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qpj5x"]
Jul 16 22:51:11.336020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.335994    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:51:11.427247 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.427208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1038b3-1f3a-4306-a464-a80a5963fe03-maas-api-service-ca\") pod \"9b1038b3-1f3a-4306-a464-a80a5963fe03\" (UID: \"9b1038b3-1f3a-4306-a464-a80a5963fe03\") "
Jul 16 22:51:11.427625 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.427605    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1038b3-1f3a-4306-a464-a80a5963fe03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b1038b3-1f3a-4306-a464-a80a5963fe03" (UID: "9b1038b3-1f3a-4306-a464-a80a5963fe03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:51:11.528918 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.528829    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b1038b3-1f3a-4306-a464-a80a5963fe03-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:51:11.602582 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:11.602547    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" path="/var/lib/kubelet/pods/eac7f40a-054c-4f60-81de-29ae18e282c5/volumes"
Jul 16 22:51:12.206754 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:12.206725    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"
Jul 16 22:51:12.207231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:12.206763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx" event={"ID":"9b1038b3-1f3a-4306-a464-a80a5963fe03","Type":"ContainerDied","Data":"0a67e974348c6e223110cd814a1fc80eb1f30a256d0007558e6fa05296c2e442"}
Jul 16 22:51:12.207231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:12.206829    2564 scope.go:117] "RemoveContainer" containerID="a34c8fb52e6a1f3c3418d0dda597fb74857cded77c1af4fef7111bb14e9949e4"
Jul 16 22:51:12.223597 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:12.223572    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"]
Jul 16 22:51:12.228458 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:12.228434    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-mtctx"]
Jul 16 22:51:13.602182 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:13.602152    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" path="/var/lib/kubelet/pods/9b1038b3-1f3a-4306-a464-a80a5963fe03/volumes"
Jul 16 22:51:19.759107 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759075    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"]
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759413    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759426    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759446    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759453    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759460    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759466    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759476    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759481    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759488    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759493    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759844 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759544    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759844 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759553    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759844 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759558    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:19.759844 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759564    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.759844 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.759572    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:19.763769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.763751    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:19.765980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.765958    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:51:19.766084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.766049    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:51:19.771042 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.771012    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"]
Jul 16 22:51:19.804587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.804554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfb93f3-8b2f-4d0c-8638-b367dc105637-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl\" (UID: \"0cfb93f3-8b2f-4d0c-8638-b367dc105637\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:19.905401 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.905356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfb93f3-8b2f-4d0c-8638-b367dc105637-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl\" (UID: \"0cfb93f3-8b2f-4d0c-8638-b367dc105637\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:19.906004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:19.905983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfb93f3-8b2f-4d0c-8638-b367dc105637-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl\" (UID: \"0cfb93f3-8b2f-4d0c-8638-b367dc105637\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:20.075826 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:20.075740    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:20.210981 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:20.210958    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"]
Jul 16 22:51:20.212873 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:51:20.212844    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfb93f3_8b2f_4d0c_8638_b367dc105637.slice/crio-3f4594c116574e1ebc24ba5e3ec7d9970305a6e6ea5438bb25dba0ff86462fe5 WatchSource:0}: Error finding container 3f4594c116574e1ebc24ba5e3ec7d9970305a6e6ea5438bb25dba0ff86462fe5: Status 404 returned error can't find the container with id 3f4594c116574e1ebc24ba5e3ec7d9970305a6e6ea5438bb25dba0ff86462fe5
Jul 16 22:51:20.252857 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:20.252820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" event={"ID":"0cfb93f3-8b2f-4d0c-8638-b367dc105637","Type":"ContainerStarted","Data":"3f4594c116574e1ebc24ba5e3ec7d9970305a6e6ea5438bb25dba0ff86462fe5"}
Jul 16 22:51:21.259192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:21.259124    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerID="31a68e9dec735197f31c9b2575d61e4d1106d78d33b672117b24bdd654f6fdcf" exitCode=6
Jul 16 22:51:21.259630 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:21.259163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" event={"ID":"0cfb93f3-8b2f-4d0c-8638-b367dc105637","Type":"ContainerDied","Data":"31a68e9dec735197f31c9b2575d61e4d1106d78d33b672117b24bdd654f6fdcf"}
Jul 16 22:51:21.259630 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:21.259441    2564 scope.go:117] "RemoveContainer" containerID="31a68e9dec735197f31c9b2575d61e4d1106d78d33b672117b24bdd654f6fdcf"
Jul 16 22:51:22.266578 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:22.266542    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerID="63903b01f718402b3e6f34038dca8e672ec0dfc13f6c60a57b1a7612c86d851e" exitCode=6
Jul 16 22:51:22.266578 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:22.266583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" event={"ID":"0cfb93f3-8b2f-4d0c-8638-b367dc105637","Type":"ContainerDied","Data":"63903b01f718402b3e6f34038dca8e672ec0dfc13f6c60a57b1a7612c86d851e"}
Jul 16 22:51:22.267071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:22.266661    2564 scope.go:117] "RemoveContainer" containerID="31a68e9dec735197f31c9b2575d61e4d1106d78d33b672117b24bdd654f6fdcf"
Jul 16 22:51:22.267071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:22.266855    2564 scope.go:117] "RemoveContainer" containerID="63903b01f718402b3e6f34038dca8e672ec0dfc13f6c60a57b1a7612c86d851e"
Jul 16 22:51:22.267221 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:22.267121    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl_odh-ai-gateway-infra(0cfb93f3-8b2f-4d0c-8638-b367dc105637)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637"
Jul 16 22:51:23.273761 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:23.273734    2564 scope.go:117] "RemoveContainer" containerID="63903b01f718402b3e6f34038dca8e672ec0dfc13f6c60a57b1a7612c86d851e"
Jul 16 22:51:23.274112 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:23.273935    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl_odh-ai-gateway-infra(0cfb93f3-8b2f-4d0c-8638-b367dc105637)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637"
Jul 16 22:51:37.596827 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:37.596794    2564 scope.go:117] "RemoveContainer" containerID="63903b01f718402b3e6f34038dca8e672ec0dfc13f6c60a57b1a7612c86d851e"
Jul 16 22:51:38.351099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:38.351062    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerID="89a35e30678d4ab0a0f98eaed769ece5b0ca590283213a14196fba1fd725b124" exitCode=6
Jul 16 22:51:38.351303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:38.351118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" event={"ID":"0cfb93f3-8b2f-4d0c-8638-b367dc105637","Type":"ContainerDied","Data":"89a35e30678d4ab0a0f98eaed769ece5b0ca590283213a14196fba1fd725b124"}
Jul 16 22:51:38.351303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:38.351199    2564 scope.go:117] "RemoveContainer" containerID="63903b01f718402b3e6f34038dca8e672ec0dfc13f6c60a57b1a7612c86d851e"
Jul 16 22:51:38.351508 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:38.351473    2564 scope.go:117] "RemoveContainer" containerID="89a35e30678d4ab0a0f98eaed769ece5b0ca590283213a14196fba1fd725b124"
Jul 16 22:51:38.351768 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:38.351741    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl_odh-ai-gateway-infra(0cfb93f3-8b2f-4d0c-8638-b367dc105637)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637"
Jul 16 22:51:39.382214 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:39.382177    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"]
Jul 16 22:51:39.516259 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:39.516235    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:39.684783 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:39.684698    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfb93f3-8b2f-4d0c-8638-b367dc105637-maas-api-service-ca\") pod \"0cfb93f3-8b2f-4d0c-8638-b367dc105637\" (UID: \"0cfb93f3-8b2f-4d0c-8638-b367dc105637\") "
Jul 16 22:51:39.685107 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:39.685086    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cfb93f3-8b2f-4d0c-8638-b367dc105637-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0cfb93f3-8b2f-4d0c-8638-b367dc105637" (UID: "0cfb93f3-8b2f-4d0c-8638-b367dc105637"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:51:39.786126 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:39.786088    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfb93f3-8b2f-4d0c-8638-b367dc105637-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:51:40.365310 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:40.365280    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"
Jul 16 22:51:40.365487 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:40.365274    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl" event={"ID":"0cfb93f3-8b2f-4d0c-8638-b367dc105637","Type":"ContainerDied","Data":"3f4594c116574e1ebc24ba5e3ec7d9970305a6e6ea5438bb25dba0ff86462fe5"}
Jul 16 22:51:40.365487 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:40.365416    2564 scope.go:117] "RemoveContainer" containerID="89a35e30678d4ab0a0f98eaed769ece5b0ca590283213a14196fba1fd725b124"
Jul 16 22:51:40.392880 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:40.388728    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"]
Jul 16 22:51:40.397083 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:40.397056    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-xc8kl"]
Jul 16 22:51:41.601303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:41.601271    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" path="/var/lib/kubelet/pods/0cfb93f3-8b2f-4d0c-8638-b367dc105637/volumes"
Jul 16 22:51:48.346513 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346419    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"]
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346786    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346797    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac7f40a-054c-4f60-81de-29ae18e282c5" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346818    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346823    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346832    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346837    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346849    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346854    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346906    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1038b3-1f3a-4306-a464-a80a5963fe03" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346916    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346922    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.346987 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.346928    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cfb93f3-8b2f-4d0c-8638-b367dc105637" containerName="revoke-keys"
Jul 16 22:51:48.351357 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.351339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:51:48.357226 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.357201    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:51:48.357397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.357273    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:51:48.362992 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.362967    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"]
Jul 16 22:51:48.465493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.465455    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a4d772-8101-4e27-96c0-ca5185d70989-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-4jmp5\" (UID: \"19a4d772-8101-4e27-96c0-ca5185d70989\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:51:48.566441 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.566389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a4d772-8101-4e27-96c0-ca5185d70989-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-4jmp5\" (UID: \"19a4d772-8101-4e27-96c0-ca5185d70989\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:51:48.567038 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.567018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a4d772-8101-4e27-96c0-ca5185d70989-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-4jmp5\" (UID: \"19a4d772-8101-4e27-96c0-ca5185d70989\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:51:48.661772 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.661666    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:51:48.795980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:48.795953    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"]
Jul 16 22:51:48.797828 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:51:48.797801    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a4d772_8101_4e27_96c0_ca5185d70989.slice/crio-e4189bd3eb2e6516b0eba4dbf736196b8d926c9a434e6fb8b6a2fec6411a4354 WatchSource:0}: Error finding container e4189bd3eb2e6516b0eba4dbf736196b8d926c9a434e6fb8b6a2fec6411a4354: Status 404 returned error can't find the container with id e4189bd3eb2e6516b0eba4dbf736196b8d926c9a434e6fb8b6a2fec6411a4354
Jul 16 22:51:49.416526 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:49.416492    2564 generic.go:356] "Generic (PLEG): container finished" podID="19a4d772-8101-4e27-96c0-ca5185d70989" containerID="4052a4807a3118be63769f21a2c78cedccac62e4d10f748ee1e00093b3f42c13" exitCode=7
Jul 16 22:51:49.416976 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:49.416578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" event={"ID":"19a4d772-8101-4e27-96c0-ca5185d70989","Type":"ContainerDied","Data":"4052a4807a3118be63769f21a2c78cedccac62e4d10f748ee1e00093b3f42c13"}
Jul 16 22:51:49.416976 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:49.416631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" event={"ID":"19a4d772-8101-4e27-96c0-ca5185d70989","Type":"ContainerStarted","Data":"e4189bd3eb2e6516b0eba4dbf736196b8d926c9a434e6fb8b6a2fec6411a4354"}
Jul 16 22:51:49.416976 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:49.416829    2564 scope.go:117] "RemoveContainer" containerID="4052a4807a3118be63769f21a2c78cedccac62e4d10f748ee1e00093b3f42c13"
Jul 16 22:51:50.423566 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:50.423476    2564 generic.go:356] "Generic (PLEG): container finished" podID="19a4d772-8101-4e27-96c0-ca5185d70989" containerID="1108a8e7d1573f7b06de88f5b5923fc07d8034af1937761fc43e5b1b7805d15b" exitCode=7
Jul 16 22:51:50.423566 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:50.423522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" event={"ID":"19a4d772-8101-4e27-96c0-ca5185d70989","Type":"ContainerDied","Data":"1108a8e7d1573f7b06de88f5b5923fc07d8034af1937761fc43e5b1b7805d15b"}
Jul 16 22:51:50.424028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:50.423574    2564 scope.go:117] "RemoveContainer" containerID="4052a4807a3118be63769f21a2c78cedccac62e4d10f748ee1e00093b3f42c13"
Jul 16 22:51:50.424028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:50.423720    2564 scope.go:117] "RemoveContainer" containerID="1108a8e7d1573f7b06de88f5b5923fc07d8034af1937761fc43e5b1b7805d15b"
Jul 16 22:51:50.424028 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:50.423921    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-4jmp5_odh-ai-gateway-infra(19a4d772-8101-4e27-96c0-ca5185d70989)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" podUID="19a4d772-8101-4e27-96c0-ca5185d70989"
Jul 16 22:51:51.429837 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.429808    2564 scope.go:117] "RemoveContainer" containerID="1108a8e7d1573f7b06de88f5b5923fc07d8034af1937761fc43e5b1b7805d15b"
Jul 16 22:51:51.430254 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:51.429994    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-4jmp5_odh-ai-gateway-infra(19a4d772-8101-4e27-96c0-ca5185d70989)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" podUID="19a4d772-8101-4e27-96c0-ca5185d70989"
Jul 16 22:51:51.745044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.745008    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"]
Jul 16 22:51:51.749477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.749459    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:51:51.761272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.761245    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"]
Jul 16 22:51:51.797971 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.797933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9891f2-bea6-425a-82e8-36dc7111303d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447\" (UID: \"8c9891f2-bea6-425a-82e8-36dc7111303d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:51:51.898942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.898885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9891f2-bea6-425a-82e8-36dc7111303d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447\" (UID: \"8c9891f2-bea6-425a-82e8-36dc7111303d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:51:51.899599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:51.899577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9891f2-bea6-425a-82e8-36dc7111303d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447\" (UID: \"8c9891f2-bea6-425a-82e8-36dc7111303d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:51:52.070499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:52.070398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:51:52.413848 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:52.413822    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"]
Jul 16 22:51:52.416280 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:51:52.416254    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9891f2_bea6_425a_82e8_36dc7111303d.slice/crio-7d3f01bdfafb6b26815a3028bcb2d9ed53846ab70f2fc60c11ab8b5a685552d3 WatchSource:0}: Error finding container 7d3f01bdfafb6b26815a3028bcb2d9ed53846ab70f2fc60c11ab8b5a685552d3: Status 404 returned error can't find the container with id 7d3f01bdfafb6b26815a3028bcb2d9ed53846ab70f2fc60c11ab8b5a685552d3
Jul 16 22:51:52.436419 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:52.436383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" event={"ID":"8c9891f2-bea6-425a-82e8-36dc7111303d","Type":"ContainerStarted","Data":"7d3f01bdfafb6b26815a3028bcb2d9ed53846ab70f2fc60c11ab8b5a685552d3"}
Jul 16 22:51:53.444091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:53.444003    2564 generic.go:356] "Generic (PLEG): container finished" podID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerID="ad7233867df2a01f7d90e8d428a5d812b8e4335ed006cf27c60a661f55aa3413" exitCode=6
Jul 16 22:51:53.444479 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:53.444087    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" event={"ID":"8c9891f2-bea6-425a-82e8-36dc7111303d","Type":"ContainerDied","Data":"ad7233867df2a01f7d90e8d428a5d812b8e4335ed006cf27c60a661f55aa3413"}
Jul 16 22:51:53.444479 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:53.444279    2564 scope.go:117] "RemoveContainer" containerID="ad7233867df2a01f7d90e8d428a5d812b8e4335ed006cf27c60a661f55aa3413"
Jul 16 22:51:54.451789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:54.451754    2564 generic.go:356] "Generic (PLEG): container finished" podID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerID="0b4d837c08d74268e6d961256300898dba8a4bceda219a527c08e5c4eca2cd6a" exitCode=6
Jul 16 22:51:54.452232 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:54.451837    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" event={"ID":"8c9891f2-bea6-425a-82e8-36dc7111303d","Type":"ContainerDied","Data":"0b4d837c08d74268e6d961256300898dba8a4bceda219a527c08e5c4eca2cd6a"}
Jul 16 22:51:54.452232 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:54.451888    2564 scope.go:117] "RemoveContainer" containerID="ad7233867df2a01f7d90e8d428a5d812b8e4335ed006cf27c60a661f55aa3413"
Jul 16 22:51:54.452232 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:54.452013    2564 scope.go:117] "RemoveContainer" containerID="0b4d837c08d74268e6d961256300898dba8a4bceda219a527c08e5c4eca2cd6a"
Jul 16 22:51:54.452361 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:54.452247    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447_odh-ai-gateway-infra(8c9891f2-bea6-425a-82e8-36dc7111303d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d"
Jul 16 22:51:55.459932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:51:55.459905    2564 scope.go:117] "RemoveContainer" containerID="0b4d837c08d74268e6d961256300898dba8a4bceda219a527c08e5c4eca2cd6a"
Jul 16 22:51:55.460407 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:51:55.460238    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447_odh-ai-gateway-infra(8c9891f2-bea6-425a-82e8-36dc7111303d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d"
Jul 16 22:52:05.600284 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:05.600249    2564 scope.go:117] "RemoveContainer" containerID="1108a8e7d1573f7b06de88f5b5923fc07d8034af1937761fc43e5b1b7805d15b"
Jul 16 22:52:06.521473 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:06.521436    2564 generic.go:356] "Generic (PLEG): container finished" podID="19a4d772-8101-4e27-96c0-ca5185d70989" containerID="57f898d279d90da31be720851469160e5f34a8a98ae3cd94d39017611279e6c7" exitCode=7
Jul 16 22:52:06.521644 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:06.521502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" event={"ID":"19a4d772-8101-4e27-96c0-ca5185d70989","Type":"ContainerDied","Data":"57f898d279d90da31be720851469160e5f34a8a98ae3cd94d39017611279e6c7"}
Jul 16 22:52:06.521644 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:06.521562    2564 scope.go:117] "RemoveContainer" containerID="1108a8e7d1573f7b06de88f5b5923fc07d8034af1937761fc43e5b1b7805d15b"
Jul 16 22:52:06.521825 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:06.521810    2564 scope.go:117] "RemoveContainer" containerID="57f898d279d90da31be720851469160e5f34a8a98ae3cd94d39017611279e6c7"
Jul 16 22:52:06.522049 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:06.522024    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-4jmp5_odh-ai-gateway-infra(19a4d772-8101-4e27-96c0-ca5185d70989)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" podUID="19a4d772-8101-4e27-96c0-ca5185d70989"
Jul 16 22:52:07.552144 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:07.552110    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"]
Jul 16 22:52:07.596462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:07.596431    2564 scope.go:117] "RemoveContainer" containerID="0b4d837c08d74268e6d961256300898dba8a4bceda219a527c08e5c4eca2cd6a"
Jul 16 22:52:07.703975 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:07.703953    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:52:07.831017 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:07.830933    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a4d772-8101-4e27-96c0-ca5185d70989-maas-api-service-ca\") pod \"19a4d772-8101-4e27-96c0-ca5185d70989\" (UID: \"19a4d772-8101-4e27-96c0-ca5185d70989\") "
Jul 16 22:52:07.831308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:07.831286    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a4d772-8101-4e27-96c0-ca5185d70989-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19a4d772-8101-4e27-96c0-ca5185d70989" (UID: "19a4d772-8101-4e27-96c0-ca5185d70989"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:52:07.932054 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:07.932018    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a4d772-8101-4e27-96c0-ca5185d70989-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:52:08.541711 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.541683    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"
Jul 16 22:52:08.541711 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.541690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5" event={"ID":"19a4d772-8101-4e27-96c0-ca5185d70989","Type":"ContainerDied","Data":"e4189bd3eb2e6516b0eba4dbf736196b8d926c9a434e6fb8b6a2fec6411a4354"}
Jul 16 22:52:08.541969 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.541772    2564 scope.go:117] "RemoveContainer" containerID="57f898d279d90da31be720851469160e5f34a8a98ae3cd94d39017611279e6c7"
Jul 16 22:52:08.543755 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.543721    2564 generic.go:356] "Generic (PLEG): container finished" podID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerID="2e678b10bef2aac8291789bd17adaffe763c7f291e4f23df632a8565925152a6" exitCode=6
Jul 16 22:52:08.543866 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.543808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" event={"ID":"8c9891f2-bea6-425a-82e8-36dc7111303d","Type":"ContainerDied","Data":"2e678b10bef2aac8291789bd17adaffe763c7f291e4f23df632a8565925152a6"}
Jul 16 22:52:08.544279 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.544262    2564 scope.go:117] "RemoveContainer" containerID="2e678b10bef2aac8291789bd17adaffe763c7f291e4f23df632a8565925152a6"
Jul 16 22:52:08.544489 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:08.544469    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447_odh-ai-gateway-infra(8c9891f2-bea6-425a-82e8-36dc7111303d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d"
Jul 16 22:52:08.550840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.550825    2564 scope.go:117] "RemoveContainer" containerID="0b4d837c08d74268e6d961256300898dba8a4bceda219a527c08e5c4eca2cd6a"
Jul 16 22:52:08.582151 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.582108    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"]
Jul 16 22:52:08.584107 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:08.584087    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4jmp5"]
Jul 16 22:52:09.574125 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:09.574085    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"]
Jul 16 22:52:09.601809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:09.601777    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" path="/var/lib/kubelet/pods/19a4d772-8101-4e27-96c0-ca5185d70989/volumes"
Jul 16 22:52:09.706813 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:09.706789    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:52:09.748359 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:09.748330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9891f2-bea6-425a-82e8-36dc7111303d-maas-api-service-ca\") pod \"8c9891f2-bea6-425a-82e8-36dc7111303d\" (UID: \"8c9891f2-bea6-425a-82e8-36dc7111303d\") "
Jul 16 22:52:09.748703 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:09.748673    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9891f2-bea6-425a-82e8-36dc7111303d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c9891f2-bea6-425a-82e8-36dc7111303d" (UID: "8c9891f2-bea6-425a-82e8-36dc7111303d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:52:09.849789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:09.849705    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9891f2-bea6-425a-82e8-36dc7111303d-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:52:10.559203 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:10.559174    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"
Jul 16 22:52:10.559402 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:10.559206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447" event={"ID":"8c9891f2-bea6-425a-82e8-36dc7111303d","Type":"ContainerDied","Data":"7d3f01bdfafb6b26815a3028bcb2d9ed53846ab70f2fc60c11ab8b5a685552d3"}
Jul 16 22:52:10.559402 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:10.559274    2564 scope.go:117] "RemoveContainer" containerID="2e678b10bef2aac8291789bd17adaffe763c7f291e4f23df632a8565925152a6"
Jul 16 22:52:10.582830 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:10.582804    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"]
Jul 16 22:52:10.591792 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:10.591765    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-h5447"]
Jul 16 22:52:11.602346 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:11.602314    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" path="/var/lib/kubelet/pods/8c9891f2-bea6-425a-82e8-36dc7111303d/volumes"
Jul 16 22:52:19.816305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816272    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"]
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816589    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816599    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816608    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816614    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816621    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816627    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816638    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816643    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816652    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816657    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816713    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816721    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816727    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:19.816757 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.816734    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:19.821491 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.821474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:19.826027 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.826001    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:52:19.826192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.826000    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:52:19.834934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.834906    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"]
Jul 16 22:52:19.932950 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:19.932913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3881486-1fe5-45d4-a2ff-d5b71d869e16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp\" (UID: \"b3881486-1fe5-45d4-a2ff-d5b71d869e16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:20.033968 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.033931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3881486-1fe5-45d4-a2ff-d5b71d869e16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp\" (UID: \"b3881486-1fe5-45d4-a2ff-d5b71d869e16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:20.034605 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.034584    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3881486-1fe5-45d4-a2ff-d5b71d869e16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp\" (UID: \"b3881486-1fe5-45d4-a2ff-d5b71d869e16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:20.132691 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.132613    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:20.270244 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.270217    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"]
Jul 16 22:52:20.272237 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:52:20.272212    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3881486_1fe5_45d4_a2ff_d5b71d869e16.slice/crio-aca2ef7c103128298f66b0ed0c982c06722a26787d3b72c5961f6753e33dbe20 WatchSource:0}: Error finding container aca2ef7c103128298f66b0ed0c982c06722a26787d3b72c5961f6753e33dbe20: Status 404 returned error can't find the container with id aca2ef7c103128298f66b0ed0c982c06722a26787d3b72c5961f6753e33dbe20
Jul 16 22:52:20.613191 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.613130    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" event={"ID":"b3881486-1fe5-45d4-a2ff-d5b71d869e16","Type":"ContainerStarted","Data":"e7d7d72b9ab0fe8802c5487354af43c28385918df85ea36466f231136c4d9c1c"}
Jul 16 22:52:20.613399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.613202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" event={"ID":"b3881486-1fe5-45d4-a2ff-d5b71d869e16","Type":"ContainerStarted","Data":"aca2ef7c103128298f66b0ed0c982c06722a26787d3b72c5961f6753e33dbe20"}
Jul 16 22:52:20.631262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:20.631213    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" podStartSLOduration=1.631197317 podStartE2EDuration="1.631197317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:52:20.629660518 +0000 UTC m=+2096.950216590" watchObservedRunningTime="2026-07-16 22:52:20.631197317 +0000 UTC m=+2096.951753388"
Jul 16 22:52:21.619702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:21.619662    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerID="e7d7d72b9ab0fe8802c5487354af43c28385918df85ea36466f231136c4d9c1c" exitCode=6
Jul 16 22:52:21.620166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:21.619720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" event={"ID":"b3881486-1fe5-45d4-a2ff-d5b71d869e16","Type":"ContainerDied","Data":"e7d7d72b9ab0fe8802c5487354af43c28385918df85ea36466f231136c4d9c1c"}
Jul 16 22:52:21.620166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:21.619979    2564 scope.go:117] "RemoveContainer" containerID="e7d7d72b9ab0fe8802c5487354af43c28385918df85ea36466f231136c4d9c1c"
Jul 16 22:52:22.626522 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:22.626483    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerID="4a77dfa8ab30d0da79b1fabcb87387ee51d31300b3f5f99c4759c48b96f2bb3a" exitCode=6
Jul 16 22:52:22.626932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:22.626547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" event={"ID":"b3881486-1fe5-45d4-a2ff-d5b71d869e16","Type":"ContainerDied","Data":"4a77dfa8ab30d0da79b1fabcb87387ee51d31300b3f5f99c4759c48b96f2bb3a"}
Jul 16 22:52:22.626932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:22.626615    2564 scope.go:117] "RemoveContainer" containerID="e7d7d72b9ab0fe8802c5487354af43c28385918df85ea36466f231136c4d9c1c"
Jul 16 22:52:22.626932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:22.626803    2564 scope.go:117] "RemoveContainer" containerID="4a77dfa8ab30d0da79b1fabcb87387ee51d31300b3f5f99c4759c48b96f2bb3a"
Jul 16 22:52:22.627065 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:22.626989    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp_odh-ai-gateway-infra(b3881486-1fe5-45d4-a2ff-d5b71d869e16)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16"
Jul 16 22:52:23.634639 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:23.634612    2564 scope.go:117] "RemoveContainer" containerID="4a77dfa8ab30d0da79b1fabcb87387ee51d31300b3f5f99c4759c48b96f2bb3a"
Jul 16 22:52:23.635020 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:23.634810    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp_odh-ai-gateway-infra(b3881486-1fe5-45d4-a2ff-d5b71d869e16)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16"
Jul 16 22:52:24.511537 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:24.511501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:52:24.521775 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:24.521749    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:52:36.596686 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:36.596648    2564 scope.go:117] "RemoveContainer" containerID="4a77dfa8ab30d0da79b1fabcb87387ee51d31300b3f5f99c4759c48b96f2bb3a"
Jul 16 22:52:37.710867 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:37.710832    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerID="16e33e0aef08c1ac33791a9da4c55c6f42f7b75652e7d9fb64d718e145c334b7" exitCode=6
Jul 16 22:52:37.711300 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:37.710886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" event={"ID":"b3881486-1fe5-45d4-a2ff-d5b71d869e16","Type":"ContainerDied","Data":"16e33e0aef08c1ac33791a9da4c55c6f42f7b75652e7d9fb64d718e145c334b7"}
Jul 16 22:52:37.711300 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:37.710931    2564 scope.go:117] "RemoveContainer" containerID="4a77dfa8ab30d0da79b1fabcb87387ee51d31300b3f5f99c4759c48b96f2bb3a"
Jul 16 22:52:37.711300 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:37.711245    2564 scope.go:117] "RemoveContainer" containerID="16e33e0aef08c1ac33791a9da4c55c6f42f7b75652e7d9fb64d718e145c334b7"
Jul 16 22:52:37.711557 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:37.711536    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp_odh-ai-gateway-infra(b3881486-1fe5-45d4-a2ff-d5b71d869e16)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16"
Jul 16 22:52:38.741600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:38.741571    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"]
Jul 16 22:52:38.901194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:38.901171    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:39.013840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.013761    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3881486-1fe5-45d4-a2ff-d5b71d869e16-maas-api-service-ca\") pod \"b3881486-1fe5-45d4-a2ff-d5b71d869e16\" (UID: \"b3881486-1fe5-45d4-a2ff-d5b71d869e16\") "
Jul 16 22:52:39.014089 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.014067    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3881486-1fe5-45d4-a2ff-d5b71d869e16-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3881486-1fe5-45d4-a2ff-d5b71d869e16" (UID: "b3881486-1fe5-45d4-a2ff-d5b71d869e16"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:52:39.115123 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.115092    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3881486-1fe5-45d4-a2ff-d5b71d869e16-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:52:39.723404 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.723367    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp" event={"ID":"b3881486-1fe5-45d4-a2ff-d5b71d869e16","Type":"ContainerDied","Data":"aca2ef7c103128298f66b0ed0c982c06722a26787d3b72c5961f6753e33dbe20"}
Jul 16 22:52:39.723404 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.723405    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"
Jul 16 22:52:39.723616 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.723440    2564 scope.go:117] "RemoveContainer" containerID="16e33e0aef08c1ac33791a9da4c55c6f42f7b75652e7d9fb64d718e145c334b7"
Jul 16 22:52:39.740947 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.740922    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"]
Jul 16 22:52:39.753341 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:39.753308    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pbjgp"]
Jul 16 22:52:41.602600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.602569    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" path="/var/lib/kubelet/pods/b3881486-1fe5-45d4-a2ff-d5b71d869e16/volumes"
Jul 16 22:52:41.783487 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783450    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"]
Jul 16 22:52:41.783797 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783785    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:41.783852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783799    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:41.783852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783816    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.783852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783822    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.783852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783835    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.783852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783840    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.784024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783892    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.784024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783900    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.784024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783908    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:41.784024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783913    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a4d772-8101-4e27-96c0-ca5185d70989" containerName="revoke-keys"
Jul 16 22:52:41.784024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.783919    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9891f2-bea6-425a-82e8-36dc7111303d" containerName="revoke-keys"
Jul 16 22:52:41.788544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.788527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:52:41.791196 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.791161    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:52:41.791196 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.791169    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:52:41.795592 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.795555    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"]
Jul 16 22:52:41.838919 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.838884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3123ad97-9087-4580-bcf1-0ecb0fc7a075-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6\" (UID: \"3123ad97-9087-4580-bcf1-0ecb0fc7a075\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:52:41.939522 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.939424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3123ad97-9087-4580-bcf1-0ecb0fc7a075-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6\" (UID: \"3123ad97-9087-4580-bcf1-0ecb0fc7a075\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:52:41.940037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:41.940015    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3123ad97-9087-4580-bcf1-0ecb0fc7a075-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6\" (UID: \"3123ad97-9087-4580-bcf1-0ecb0fc7a075\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:52:42.101385 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:42.101354    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:52:42.231277 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:42.231252    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"]
Jul 16 22:52:42.232909 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:52:42.232879    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3123ad97_9087_4580_bcf1_0ecb0fc7a075.slice/crio-0c99f5d3e248fbc2c770941f375c19a1a041f478cc083dc72111e7b719591fce WatchSource:0}: Error finding container 0c99f5d3e248fbc2c770941f375c19a1a041f478cc083dc72111e7b719591fce: Status 404 returned error can't find the container with id 0c99f5d3e248fbc2c770941f375c19a1a041f478cc083dc72111e7b719591fce
Jul 16 22:52:42.741739 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:42.741643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerStarted","Data":"26d06691c92f35b8f626fdf5c60d000c7d2e005426ec3ab3d3be02a5d5c20c1d"}
Jul 16 22:52:42.741739 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:42.741699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerStarted","Data":"0c99f5d3e248fbc2c770941f375c19a1a041f478cc083dc72111e7b719591fce"}
Jul 16 22:52:42.759477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:42.759419    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" podStartSLOduration=1.759403064 podStartE2EDuration="1.759403064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:52:42.757041834 +0000 UTC m=+2119.077597906" watchObservedRunningTime="2026-07-16 22:52:42.759403064 +0000 UTC m=+2119.079959274"
Jul 16 22:52:43.747754 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:43.747718    2564 generic.go:356] "Generic (PLEG): container finished" podID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerID="26d06691c92f35b8f626fdf5c60d000c7d2e005426ec3ab3d3be02a5d5c20c1d" exitCode=6
Jul 16 22:52:43.748194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:43.747813    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerDied","Data":"26d06691c92f35b8f626fdf5c60d000c7d2e005426ec3ab3d3be02a5d5c20c1d"}
Jul 16 22:52:43.748194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:43.748000    2564 scope.go:117] "RemoveContainer" containerID="26d06691c92f35b8f626fdf5c60d000c7d2e005426ec3ab3d3be02a5d5c20c1d"
Jul 16 22:52:44.754474 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:44.754393    2564 generic.go:356] "Generic (PLEG): container finished" podID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerID="05c3e901ac02277e660949ddb38964ce0af9fb2681b551bcb4e5c088d7c7ba20" exitCode=6
Jul 16 22:52:44.754916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:44.754488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerDied","Data":"05c3e901ac02277e660949ddb38964ce0af9fb2681b551bcb4e5c088d7c7ba20"}
Jul 16 22:52:44.754916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:44.754550    2564 scope.go:117] "RemoveContainer" containerID="26d06691c92f35b8f626fdf5c60d000c7d2e005426ec3ab3d3be02a5d5c20c1d"
Jul 16 22:52:44.754916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:44.754666    2564 scope.go:117] "RemoveContainer" containerID="05c3e901ac02277e660949ddb38964ce0af9fb2681b551bcb4e5c088d7c7ba20"
Jul 16 22:52:44.754916 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:44.754897    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6_odh-ai-gateway-infra(3123ad97-9087-4580-bcf1-0ecb0fc7a075)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075"
Jul 16 22:52:45.763078 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:45.763050    2564 scope.go:117] "RemoveContainer" containerID="05c3e901ac02277e660949ddb38964ce0af9fb2681b551bcb4e5c088d7c7ba20"
Jul 16 22:52:45.763491 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:45.763276    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6_odh-ai-gateway-infra(3123ad97-9087-4580-bcf1-0ecb0fc7a075)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075"
Jul 16 22:52:48.400111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.400077    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"]
Jul 16 22:52:48.400507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.400443    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:48.400507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.400454    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3881486-1fe5-45d4-a2ff-d5b71d869e16" containerName="revoke-keys"
Jul 16 22:52:48.403528 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.403511    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:52:48.413530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.413505    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"]
Jul 16 22:52:48.486754 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.486717    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ec1d08-4581-4072-97f7-a79eb2fb01d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-j9j95\" (UID: \"c5ec1d08-4581-4072-97f7-a79eb2fb01d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:52:48.587650 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.587608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ec1d08-4581-4072-97f7-a79eb2fb01d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-j9j95\" (UID: \"c5ec1d08-4581-4072-97f7-a79eb2fb01d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:52:48.588245 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.588225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ec1d08-4581-4072-97f7-a79eb2fb01d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-j9j95\" (UID: \"c5ec1d08-4581-4072-97f7-a79eb2fb01d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:52:48.714861 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.714826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:52:48.845463 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:48.845436    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"]
Jul 16 22:52:48.847491 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:52:48.847449    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ec1d08_4581_4072_97f7_a79eb2fb01d4.slice/crio-3647feab1e21d81e2db54044ea9e6c3f0a011e0442a0072dd0b43e6c24d3106d WatchSource:0}: Error finding container 3647feab1e21d81e2db54044ea9e6c3f0a011e0442a0072dd0b43e6c24d3106d: Status 404 returned error can't find the container with id 3647feab1e21d81e2db54044ea9e6c3f0a011e0442a0072dd0b43e6c24d3106d
Jul 16 22:52:49.786464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:49.786420    2564 generic.go:356] "Generic (PLEG): container finished" podID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerID="bd77005c5497b1237bc4ec3c37d6e70e82b5c4ebbef6d48c75c9ea417278973f" exitCode=7
Jul 16 22:52:49.786911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:49.786505    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" event={"ID":"c5ec1d08-4581-4072-97f7-a79eb2fb01d4","Type":"ContainerDied","Data":"bd77005c5497b1237bc4ec3c37d6e70e82b5c4ebbef6d48c75c9ea417278973f"}
Jul 16 22:52:49.786911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:49.786561    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" event={"ID":"c5ec1d08-4581-4072-97f7-a79eb2fb01d4","Type":"ContainerStarted","Data":"3647feab1e21d81e2db54044ea9e6c3f0a011e0442a0072dd0b43e6c24d3106d"}
Jul 16 22:52:49.786911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:49.786709    2564 scope.go:117] "RemoveContainer" containerID="bd77005c5497b1237bc4ec3c37d6e70e82b5c4ebbef6d48c75c9ea417278973f"
Jul 16 22:52:50.794358 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:50.794266    2564 generic.go:356] "Generic (PLEG): container finished" podID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerID="c78d2ce508c63f275c482dc8e1af05b435884b8acc274cd1122eb78f0e432d98" exitCode=7
Jul 16 22:52:50.794847 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:50.794356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" event={"ID":"c5ec1d08-4581-4072-97f7-a79eb2fb01d4","Type":"ContainerDied","Data":"c78d2ce508c63f275c482dc8e1af05b435884b8acc274cd1122eb78f0e432d98"}
Jul 16 22:52:50.794847 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:50.794413    2564 scope.go:117] "RemoveContainer" containerID="bd77005c5497b1237bc4ec3c37d6e70e82b5c4ebbef6d48c75c9ea417278973f"
Jul 16 22:52:50.794847 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:50.794575    2564 scope.go:117] "RemoveContainer" containerID="c78d2ce508c63f275c482dc8e1af05b435884b8acc274cd1122eb78f0e432d98"
Jul 16 22:52:50.794847 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:50.794797    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-j9j95_odh-ai-gateway-infra(c5ec1d08-4581-4072-97f7-a79eb2fb01d4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4"
Jul 16 22:52:51.801171 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:51.801126    2564 scope.go:117] "RemoveContainer" containerID="c78d2ce508c63f275c482dc8e1af05b435884b8acc274cd1122eb78f0e432d98"
Jul 16 22:52:51.801547 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:52:51.801331    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-j9j95_odh-ai-gateway-infra(c5ec1d08-4581-4072-97f7-a79eb2fb01d4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4"
Jul 16 22:52:59.597018 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:59.596986    2564 scope.go:117] "RemoveContainer" containerID="05c3e901ac02277e660949ddb38964ce0af9fb2681b551bcb4e5c088d7c7ba20"
Jul 16 22:52:59.846662 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:52:59.846632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerStarted","Data":"13e61345726acfe7167ebd83ff18de328eb95a39e4dd04f9afcf902ca0b2b5ae"}
Jul 16 22:53:00.853028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:00.852997    2564 generic.go:356] "Generic (PLEG): container finished" podID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerID="13e61345726acfe7167ebd83ff18de328eb95a39e4dd04f9afcf902ca0b2b5ae" exitCode=6
Jul 16 22:53:00.853440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:00.853075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerDied","Data":"13e61345726acfe7167ebd83ff18de328eb95a39e4dd04f9afcf902ca0b2b5ae"}
Jul 16 22:53:00.853440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:00.853148    2564 scope.go:117] "RemoveContainer" containerID="05c3e901ac02277e660949ddb38964ce0af9fb2681b551bcb4e5c088d7c7ba20"
Jul 16 22:53:00.853440 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:00.853401    2564 scope.go:117] "RemoveContainer" containerID="13e61345726acfe7167ebd83ff18de328eb95a39e4dd04f9afcf902ca0b2b5ae"
Jul 16 22:53:00.853638 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:00.853620    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6_odh-ai-gateway-infra(3123ad97-9087-4580-bcf1-0ecb0fc7a075)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075"
Jul 16 22:53:00.886660 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:00.886629    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"]
Jul 16 22:53:01.995263 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:01.995237    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:53:02.107391 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.107356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3123ad97-9087-4580-bcf1-0ecb0fc7a075-maas-api-service-ca\") pod \"3123ad97-9087-4580-bcf1-0ecb0fc7a075\" (UID: \"3123ad97-9087-4580-bcf1-0ecb0fc7a075\") "
Jul 16 22:53:02.107727 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.107703    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3123ad97-9087-4580-bcf1-0ecb0fc7a075-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3123ad97-9087-4580-bcf1-0ecb0fc7a075" (UID: "3123ad97-9087-4580-bcf1-0ecb0fc7a075"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:53:02.207885 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.207858    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3123ad97-9087-4580-bcf1-0ecb0fc7a075-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:02.867918 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.867892    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"
Jul 16 22:53:02.868094 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.867925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6" event={"ID":"3123ad97-9087-4580-bcf1-0ecb0fc7a075","Type":"ContainerDied","Data":"0c99f5d3e248fbc2c770941f375c19a1a041f478cc083dc72111e7b719591fce"}
Jul 16 22:53:02.868094 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.867989    2564 scope.go:117] "RemoveContainer" containerID="13e61345726acfe7167ebd83ff18de328eb95a39e4dd04f9afcf902ca0b2b5ae"
Jul 16 22:53:02.894715 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.894685    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"]
Jul 16 22:53:02.899990 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:02.899963    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-w6rq6"]
Jul 16 22:53:03.602308 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:03.602269    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" path="/var/lib/kubelet/pods/3123ad97-9087-4580-bcf1-0ecb0fc7a075/volumes"
Jul 16 22:53:06.596746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:06.596715    2564 scope.go:117] "RemoveContainer" containerID="c78d2ce508c63f275c482dc8e1af05b435884b8acc274cd1122eb78f0e432d98"
Jul 16 22:53:06.891662 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:06.891573    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" event={"ID":"c5ec1d08-4581-4072-97f7-a79eb2fb01d4","Type":"ContainerStarted","Data":"24ce7a301eaf2878f16339fc95f3a426639ab134a61a2169166d43c345d0ea79"}
Jul 16 22:53:06.908847 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:06.908689    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" podStartSLOduration=18.908671715 podStartE2EDuration="18.908671715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:53:06.907688755 +0000 UTC m=+2143.228244826" watchObservedRunningTime="2026-07-16 22:53:06.908671715 +0000 UTC m=+2143.229227786"
Jul 16 22:53:07.898623 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:07.898589    2564 generic.go:356] "Generic (PLEG): container finished" podID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerID="24ce7a301eaf2878f16339fc95f3a426639ab134a61a2169166d43c345d0ea79" exitCode=7
Jul 16 22:53:07.899077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:07.898638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" event={"ID":"c5ec1d08-4581-4072-97f7-a79eb2fb01d4","Type":"ContainerDied","Data":"24ce7a301eaf2878f16339fc95f3a426639ab134a61a2169166d43c345d0ea79"}
Jul 16 22:53:07.899077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:07.898696    2564 scope.go:117] "RemoveContainer" containerID="c78d2ce508c63f275c482dc8e1af05b435884b8acc274cd1122eb78f0e432d98"
Jul 16 22:53:07.899077 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:07.898970    2564 scope.go:117] "RemoveContainer" containerID="24ce7a301eaf2878f16339fc95f3a426639ab134a61a2169166d43c345d0ea79"
Jul 16 22:53:07.899258 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:07.899202    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-j9j95_odh-ai-gateway-infra(c5ec1d08-4581-4072-97f7-a79eb2fb01d4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4"
Jul 16 22:53:07.927816 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:07.927787    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"]
Jul 16 22:53:09.037094 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.037072    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:53:09.166665 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.166569    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ec1d08-4581-4072-97f7-a79eb2fb01d4-maas-api-service-ca\") pod \"c5ec1d08-4581-4072-97f7-a79eb2fb01d4\" (UID: \"c5ec1d08-4581-4072-97f7-a79eb2fb01d4\") "
Jul 16 22:53:09.166940 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.166917    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ec1d08-4581-4072-97f7-a79eb2fb01d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5ec1d08-4581-4072-97f7-a79eb2fb01d4" (UID: "c5ec1d08-4581-4072-97f7-a79eb2fb01d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:53:09.267575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.267536    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ec1d08-4581-4072-97f7-a79eb2fb01d4-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:09.860226 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860129    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"]
Jul 16 22:53:09.860507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860493    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860508    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860516    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860521    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860529    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860534    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860544    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860549    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860557    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860562    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860618    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860625    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860631    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860637    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3123ad97-9087-4580-bcf1-0ecb0fc7a075" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860644    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.860809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.860652    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:09.863578 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.863562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:09.871206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.871175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"]
Jul 16 22:53:09.911840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.911808    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"
Jul 16 22:53:09.912035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.911858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95" event={"ID":"c5ec1d08-4581-4072-97f7-a79eb2fb01d4","Type":"ContainerDied","Data":"3647feab1e21d81e2db54044ea9e6c3f0a011e0442a0072dd0b43e6c24d3106d"}
Jul 16 22:53:09.912035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.911909    2564 scope.go:117] "RemoveContainer" containerID="24ce7a301eaf2878f16339fc95f3a426639ab134a61a2169166d43c345d0ea79"
Jul 16 22:53:09.950337 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.950309    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"]
Jul 16 22:53:09.964477 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.964448    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-j9j95"]
Jul 16 22:53:09.973307 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:09.973283    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9afe99ab-84b2-4afb-bb31-342e8a1168cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq\" (UID: \"9afe99ab-84b2-4afb-bb31-342e8a1168cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:10.074766 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.074726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9afe99ab-84b2-4afb-bb31-342e8a1168cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq\" (UID: \"9afe99ab-84b2-4afb-bb31-342e8a1168cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:10.075393 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.075375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9afe99ab-84b2-4afb-bb31-342e8a1168cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq\" (UID: \"9afe99ab-84b2-4afb-bb31-342e8a1168cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:10.176217 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.176105    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:10.309408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.309379    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"]
Jul 16 22:53:10.311288 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:53:10.311261    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afe99ab_84b2_4afb_bb31_342e8a1168cc.slice/crio-d34a7433c1330e82c32c7bc2fc746e5683f6a418bce0b988e150035b2db88884 WatchSource:0}: Error finding container d34a7433c1330e82c32c7bc2fc746e5683f6a418bce0b988e150035b2db88884: Status 404 returned error can't find the container with id d34a7433c1330e82c32c7bc2fc746e5683f6a418bce0b988e150035b2db88884
Jul 16 22:53:10.919575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.919540    2564 generic.go:356] "Generic (PLEG): container finished" podID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerID="bc21b05779c3f33bfd8a145da4d73ce01acb9e59d8667b9f979d3796481b1d4f" exitCode=6
Jul 16 22:53:10.919716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.919621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" event={"ID":"9afe99ab-84b2-4afb-bb31-342e8a1168cc","Type":"ContainerDied","Data":"bc21b05779c3f33bfd8a145da4d73ce01acb9e59d8667b9f979d3796481b1d4f"}
Jul 16 22:53:10.919716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.919702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" event={"ID":"9afe99ab-84b2-4afb-bb31-342e8a1168cc","Type":"ContainerStarted","Data":"d34a7433c1330e82c32c7bc2fc746e5683f6a418bce0b988e150035b2db88884"}
Jul 16 22:53:10.919855 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:10.919838    2564 scope.go:117] "RemoveContainer" containerID="bc21b05779c3f33bfd8a145da4d73ce01acb9e59d8667b9f979d3796481b1d4f"
Jul 16 22:53:11.602052 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:11.602021    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" path="/var/lib/kubelet/pods/c5ec1d08-4581-4072-97f7-a79eb2fb01d4/volumes"
Jul 16 22:53:11.926152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:11.926039    2564 generic.go:356] "Generic (PLEG): container finished" podID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerID="9b012fa169e3d93802a02c723ef05686e9118c8fcab958a02c69d9cbb431c640" exitCode=6
Jul 16 22:53:11.926152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:11.926124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" event={"ID":"9afe99ab-84b2-4afb-bb31-342e8a1168cc","Type":"ContainerDied","Data":"9b012fa169e3d93802a02c723ef05686e9118c8fcab958a02c69d9cbb431c640"}
Jul 16 22:53:11.926386 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:11.926202    2564 scope.go:117] "RemoveContainer" containerID="bc21b05779c3f33bfd8a145da4d73ce01acb9e59d8667b9f979d3796481b1d4f"
Jul 16 22:53:11.926386 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:11.926350    2564 scope.go:117] "RemoveContainer" containerID="9b012fa169e3d93802a02c723ef05686e9118c8fcab958a02c69d9cbb431c640"
Jul 16 22:53:11.926634 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:11.926611    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq_odh-ai-gateway-infra(9afe99ab-84b2-4afb-bb31-342e8a1168cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc"
Jul 16 22:53:12.933556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:12.933525    2564 scope.go:117] "RemoveContainer" containerID="9b012fa169e3d93802a02c723ef05686e9118c8fcab958a02c69d9cbb431c640"
Jul 16 22:53:12.933941 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:12.933753    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq_odh-ai-gateway-infra(9afe99ab-84b2-4afb-bb31-342e8a1168cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc"
Jul 16 22:53:24.596997 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:24.596913    2564 scope.go:117] "RemoveContainer" containerID="9b012fa169e3d93802a02c723ef05686e9118c8fcab958a02c69d9cbb431c640"
Jul 16 22:53:24.650550 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:24.650520    2564 scope.go:117] "RemoveContainer" containerID="f6c662ae5d8c82c0de161ac604713004e8ff7c64e3e940d5c9106bba011fa0fc"
Jul 16 22:53:24.995455 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:24.995419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" event={"ID":"9afe99ab-84b2-4afb-bb31-342e8a1168cc","Type":"ContainerStarted","Data":"dccd51dbf2617f7dfc32796ccafb18c393e10a7318ee4384360959dc9066140f"}
Jul 16 22:53:25.019624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:25.019571    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" podStartSLOduration=16.019554543 podStartE2EDuration="16.019554543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:53:25.016941519 +0000 UTC m=+2161.337497602" watchObservedRunningTime="2026-07-16 22:53:25.019554543 +0000 UTC m=+2161.340110623"
Jul 16 22:53:26.001928 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:26.001897    2564 generic.go:356] "Generic (PLEG): container finished" podID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerID="dccd51dbf2617f7dfc32796ccafb18c393e10a7318ee4384360959dc9066140f" exitCode=6
Jul 16 22:53:26.002356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:26.001979    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" event={"ID":"9afe99ab-84b2-4afb-bb31-342e8a1168cc","Type":"ContainerDied","Data":"dccd51dbf2617f7dfc32796ccafb18c393e10a7318ee4384360959dc9066140f"}
Jul 16 22:53:26.002356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:26.002046    2564 scope.go:117] "RemoveContainer" containerID="9b012fa169e3d93802a02c723ef05686e9118c8fcab958a02c69d9cbb431c640"
Jul 16 22:53:26.002356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:26.002320    2564 scope.go:117] "RemoveContainer" containerID="dccd51dbf2617f7dfc32796ccafb18c393e10a7318ee4384360959dc9066140f"
Jul 16 22:53:26.002577 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:26.002559    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq_odh-ai-gateway-infra(9afe99ab-84b2-4afb-bb31-342e8a1168cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc"
Jul 16 22:53:26.047924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:26.047892    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"]
Jul 16 22:53:27.143037 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:27.143008    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:27.224950 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:27.224916    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9afe99ab-84b2-4afb-bb31-342e8a1168cc-maas-api-service-ca\") pod \"9afe99ab-84b2-4afb-bb31-342e8a1168cc\" (UID: \"9afe99ab-84b2-4afb-bb31-342e8a1168cc\") "
Jul 16 22:53:27.225305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:27.225281    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9afe99ab-84b2-4afb-bb31-342e8a1168cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9afe99ab-84b2-4afb-bb31-342e8a1168cc" (UID: "9afe99ab-84b2-4afb-bb31-342e8a1168cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:53:27.326618 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:27.326532    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9afe99ab-84b2-4afb-bb31-342e8a1168cc-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:28.015480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:28.015449    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"
Jul 16 22:53:28.015659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:28.015499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq" event={"ID":"9afe99ab-84b2-4afb-bb31-342e8a1168cc","Type":"ContainerDied","Data":"d34a7433c1330e82c32c7bc2fc746e5683f6a418bce0b988e150035b2db88884"}
Jul 16 22:53:28.015659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:28.015592    2564 scope.go:117] "RemoveContainer" containerID="dccd51dbf2617f7dfc32796ccafb18c393e10a7318ee4384360959dc9066140f"
Jul 16 22:53:28.031922 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:28.031891    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"]
Jul 16 22:53:28.035788 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:28.035760    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mvsbq"]
Jul 16 22:53:29.602862 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:29.602830    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" path="/var/lib/kubelet/pods/9afe99ab-84b2-4afb-bb31-342e8a1168cc/volumes"
Jul 16 22:53:41.911411 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911372    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"]
Jul 16 22:53:41.911913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911837    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.911913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911856    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.911913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911889    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:41.911913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911897    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ec1d08-4581-4072-97f7-a79eb2fb01d4" containerName="revoke-keys"
Jul 16 22:53:41.911913 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911912    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.912204 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911921    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.912204 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911930    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.912204 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.911938    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.912204 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.912023    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.912204 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.912035    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:41.916521 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.916499    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:53:41.919387 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.919368    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:53:41.919462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.919405    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:53:41.923411 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.923377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"]
Jul 16 22:53:41.959377 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:41.959350    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828544e4-dd4a-464b-b6ba-c521ab4d5217-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt\" (UID: \"828544e4-dd4a-464b-b6ba-c521ab4d5217\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:53:42.060862 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:42.060829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828544e4-dd4a-464b-b6ba-c521ab4d5217-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt\" (UID: \"828544e4-dd4a-464b-b6ba-c521ab4d5217\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:53:42.061443 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:42.061423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828544e4-dd4a-464b-b6ba-c521ab4d5217-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt\" (UID: \"828544e4-dd4a-464b-b6ba-c521ab4d5217\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:53:42.228626 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:42.228583    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:53:42.362098 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:42.362067    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"]
Jul 16 22:53:42.363672 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:53:42.363646    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828544e4_dd4a_464b_b6ba_c521ab4d5217.slice/crio-43d8dad2784ae01935a4e044666ed18a68d3f4617b23debcd65603ede2e251a9 WatchSource:0}: Error finding container 43d8dad2784ae01935a4e044666ed18a68d3f4617b23debcd65603ede2e251a9: Status 404 returned error can't find the container with id 43d8dad2784ae01935a4e044666ed18a68d3f4617b23debcd65603ede2e251a9
Jul 16 22:53:43.093250 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:43.093211    2564 generic.go:356] "Generic (PLEG): container finished" podID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerID="7366377643f8fa76c155f35ea9d2a27c9c7d9c55a05ecdd09d7af707c8253c26" exitCode=6
Jul 16 22:53:43.093631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:43.093277    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" event={"ID":"828544e4-dd4a-464b-b6ba-c521ab4d5217","Type":"ContainerDied","Data":"7366377643f8fa76c155f35ea9d2a27c9c7d9c55a05ecdd09d7af707c8253c26"}
Jul 16 22:53:43.093631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:43.093328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" event={"ID":"828544e4-dd4a-464b-b6ba-c521ab4d5217","Type":"ContainerStarted","Data":"43d8dad2784ae01935a4e044666ed18a68d3f4617b23debcd65603ede2e251a9"}
Jul 16 22:53:43.093631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:43.093542    2564 scope.go:117] "RemoveContainer" containerID="7366377643f8fa76c155f35ea9d2a27c9c7d9c55a05ecdd09d7af707c8253c26"
Jul 16 22:53:44.101302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:44.101260    2564 generic.go:356] "Generic (PLEG): container finished" podID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerID="3a23c761f65afbdc638cb538f3cd7ebbe1ee9f48bc94f3600bffd296f21ef072" exitCode=6
Jul 16 22:53:44.101752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:44.101350    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" event={"ID":"828544e4-dd4a-464b-b6ba-c521ab4d5217","Type":"ContainerDied","Data":"3a23c761f65afbdc638cb538f3cd7ebbe1ee9f48bc94f3600bffd296f21ef072"}
Jul 16 22:53:44.101752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:44.101406    2564 scope.go:117] "RemoveContainer" containerID="7366377643f8fa76c155f35ea9d2a27c9c7d9c55a05ecdd09d7af707c8253c26"
Jul 16 22:53:44.101752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:44.101637    2564 scope.go:117] "RemoveContainer" containerID="3a23c761f65afbdc638cb538f3cd7ebbe1ee9f48bc94f3600bffd296f21ef072"
Jul 16 22:53:44.101867 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:44.101847    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt_odh-ai-gateway-infra(828544e4-dd4a-464b-b6ba-c521ab4d5217)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217"
Jul 16 22:53:45.108309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:45.108280    2564 scope.go:117] "RemoveContainer" containerID="3a23c761f65afbdc638cb538f3cd7ebbe1ee9f48bc94f3600bffd296f21ef072"
Jul 16 22:53:45.108694 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:45.108510    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt_odh-ai-gateway-infra(828544e4-dd4a-464b-b6ba-c521ab4d5217)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217"
Jul 16 22:53:48.378192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.378153    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"]
Jul 16 22:53:48.378618 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.378413    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy" containerID="cri-o://6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992" gracePeriod=30
Jul 16 22:53:48.458002 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.457961    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"]
Jul 16 22:53:48.458419 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.458404    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afe99ab-84b2-4afb-bb31-342e8a1168cc" containerName="revoke-keys"
Jul 16 22:53:48.462593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.462568    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:53:48.474219 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.474191    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"]
Jul 16 22:53:48.519385 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.519345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b41c74-465f-4483-8319-d719d40a317a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-z87s6\" (UID: \"25b41c74-465f-4483-8319-d719d40a317a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:53:48.620241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.620194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b41c74-465f-4483-8319-d719d40a317a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-z87s6\" (UID: \"25b41c74-465f-4483-8319-d719d40a317a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:53:48.620872 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.620850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b41c74-465f-4483-8319-d719d40a317a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-z87s6\" (UID: \"25b41c74-465f-4483-8319-d719d40a317a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:53:48.772541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.772499    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:53:48.912015 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:48.911987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"]
Jul 16 22:53:48.913949 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:53:48.913919    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b41c74_465f_4483_8319_d719d40a317a.slice/crio-a431514f556254536e2721948fc34647dac2d0f92f4d2a7ee4eb9acde3e26af5 WatchSource:0}: Error finding container a431514f556254536e2721948fc34647dac2d0f92f4d2a7ee4eb9acde3e26af5: Status 404 returned error can't find the container with id a431514f556254536e2721948fc34647dac2d0f92f4d2a7ee4eb9acde3e26af5
Jul 16 22:53:49.129789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:49.129758    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerStarted","Data":"5892099fc03d1f527b16070508ac27f5573a8d54d2c7d81c30f837fa23c68dd6"}
Jul 16 22:53:49.129988 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:49.129808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerStarted","Data":"a431514f556254536e2721948fc34647dac2d0f92f4d2a7ee4eb9acde3e26af5"}
Jul 16 22:53:49.148621 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:49.148574    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" podStartSLOduration=1.148560836 podStartE2EDuration="1.148560836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:53:49.146423123 +0000 UTC m=+2185.466979195" watchObservedRunningTime="2026-07-16 22:53:49.148560836 +0000 UTC m=+2185.469116946"
Jul 16 22:53:50.136147 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:50.136101    2564 generic.go:356] "Generic (PLEG): container finished" podID="25b41c74-465f-4483-8319-d719d40a317a" containerID="5892099fc03d1f527b16070508ac27f5573a8d54d2c7d81c30f837fa23c68dd6" exitCode=7
Jul 16 22:53:50.136534 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:50.136175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerDied","Data":"5892099fc03d1f527b16070508ac27f5573a8d54d2c7d81c30f837fa23c68dd6"}
Jul 16 22:53:50.136534 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:50.136329    2564 scope.go:117] "RemoveContainer" containerID="5892099fc03d1f527b16070508ac27f5573a8d54d2c7d81c30f837fa23c68dd6"
Jul 16 22:53:51.143299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:51.143212    2564 generic.go:356] "Generic (PLEG): container finished" podID="25b41c74-465f-4483-8319-d719d40a317a" containerID="8c84af0f99ca344aa753112a088dc1cb84298d102ec24049a803dcf55d2e037b" exitCode=7
Jul 16 22:53:51.143299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:51.143273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerDied","Data":"8c84af0f99ca344aa753112a088dc1cb84298d102ec24049a803dcf55d2e037b"}
Jul 16 22:53:51.143819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:51.143347    2564 scope.go:117] "RemoveContainer" containerID="5892099fc03d1f527b16070508ac27f5573a8d54d2c7d81c30f837fa23c68dd6"
Jul 16 22:53:51.143819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:51.143565    2564 scope.go:117] "RemoveContainer" containerID="8c84af0f99ca344aa753112a088dc1cb84298d102ec24049a803dcf55d2e037b"
Jul 16 22:53:51.143923 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:51.143809    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-z87s6_odh-ai-gateway-infra(25b41c74-465f-4483-8319-d719d40a317a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" podUID="25b41c74-465f-4483-8319-d719d40a317a"
Jul 16 22:53:51.189391 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:51.189357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:53:52.150347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:52.150322    2564 scope.go:117] "RemoveContainer" containerID="8c84af0f99ca344aa753112a088dc1cb84298d102ec24049a803dcf55d2e037b"
Jul 16 22:53:52.150721 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:52.150498    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-z87s6_odh-ai-gateway-infra(25b41c74-465f-4483-8319-d719d40a317a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" podUID="25b41c74-465f-4483-8319-d719d40a317a"
Jul 16 22:53:53.627461 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.627436    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:53:53.663840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.663736    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-socket\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.663840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.663784    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-data\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.663840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.663813    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06da0dba-b829-4539-a577-c43a8a3f336d-istiod-ca-cert\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.664166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.663863    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-envoy\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.664166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.663927    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzhkr\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-kube-api-access-dzhkr\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.664166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.663972    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-certs\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.664166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.664010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-credential-socket\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.664407 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.664379    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:53:53.664472 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.664450    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06da0dba-b829-4539-a577-c43a8a3f336d-istio-podinfo\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.664702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.664662    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:53:53.665234 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.664876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-istio-token\") pod \"06da0dba-b829-4539-a577-c43a8a3f336d\" (UID: \"06da0dba-b829-4539-a577-c43a8a3f336d\") "
Jul 16 22:53:53.666304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.665712    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.666304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.665736    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.666304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.664945    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-data" (OuterVolumeSpecName: "istio-data") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:53:53.666304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.665088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06da0dba-b829-4539-a577-c43a8a3f336d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:53:53.666304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.665219    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:53:53.667049 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.667008    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06da0dba-b829-4539-a577-c43a8a3f336d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 22:53:53.667164 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.667088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:53:53.667489 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.667457    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-kube-api-access-dzhkr" (OuterVolumeSpecName: "kube-api-access-dzhkr") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "kube-api-access-dzhkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:53:53.668828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.668808    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-istio-token" (OuterVolumeSpecName: "istio-token") pod "06da0dba-b829-4539-a577-c43a8a3f336d" (UID: "06da0dba-b829-4539-a577-c43a8a3f336d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:53:53.766336 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766298    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzhkr\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-kube-api-access-dzhkr\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.766336 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766333    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06da0dba-b829-4539-a577-c43a8a3f336d-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.766544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766346    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06da0dba-b829-4539-a577-c43a8a3f336d-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.766544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766358    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.766544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766368    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.766544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766380    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06da0dba-b829-4539-a577-c43a8a3f336d-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:53.766544 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:53.766390    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06da0dba-b829-4539-a577-c43a8a3f336d-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:54.162906 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.162874    2564 generic.go:356] "Generic (PLEG): container finished" podID="06da0dba-b829-4539-a577-c43a8a3f336d" containerID="6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992" exitCode=0
Jul 16 22:53:54.163088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.162921    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"
Jul 16 22:53:54.163088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.162965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" event={"ID":"06da0dba-b829-4539-a577-c43a8a3f336d","Type":"ContainerDied","Data":"6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992"}
Jul 16 22:53:54.163088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.163019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6" event={"ID":"06da0dba-b829-4539-a577-c43a8a3f336d","Type":"ContainerDied","Data":"af2df2f5743363a94397296e6dee81e35d8c36abe74d7c64d83b2d4e9fff1b54"}
Jul 16 22:53:54.163088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.163050    2564 scope.go:117] "RemoveContainer" containerID="6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992"
Jul 16 22:53:54.172970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.172950    2564 scope.go:117] "RemoveContainer" containerID="6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"
Jul 16 22:53:54.181103 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.181084    2564 scope.go:117] "RemoveContainer" containerID="6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992"
Jul 16 22:53:54.181383 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:54.181364    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992\": container with ID starting with 6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992 not found: ID does not exist" containerID="6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992"
Jul 16 22:53:54.181451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.181392    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992"} err="failed to get container status \"6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992\": rpc error: code = NotFound desc = could not find container \"6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992\": container with ID starting with 6e9f1915177f76972fe9e21050e2caa13c6d273ffa62d148681fbc3cd8425992 not found: ID does not exist"
Jul 16 22:53:54.181451 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.181411    2564 scope.go:117] "RemoveContainer" containerID="6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"
Jul 16 22:53:54.181663 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:54.181646    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e\": container with ID starting with 6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e not found: ID does not exist" containerID="6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"
Jul 16 22:53:54.181700 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.181670    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e"} err="failed to get container status \"6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e\": rpc error: code = NotFound desc = could not find container \"6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e\": container with ID starting with 6bb9907422b4ab1a434c221e891e8e6455cb022e35f0dace5f42532b2009214e not found: ID does not exist"
Jul 16 22:53:54.186064 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.186040    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"]
Jul 16 22:53:54.190571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:54.190549    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-234edc78-openshift-default-79db47cb99-2s5p6"]
Jul 16 22:53:55.602148 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:55.602106    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" path="/var/lib/kubelet/pods/06da0dba-b829-4539-a577-c43a8a3f336d/volumes"
Jul 16 22:53:56.596201 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:56.596168    2564 scope.go:117] "RemoveContainer" containerID="3a23c761f65afbdc638cb538f3cd7ebbe1ee9f48bc94f3600bffd296f21ef072"
Jul 16 22:53:57.182193 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:57.182158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" event={"ID":"828544e4-dd4a-464b-b6ba-c521ab4d5217","Type":"ContainerStarted","Data":"1c51c139dd9db2b8e69a3b305aad910128b9f958a55e6a93ec4ccbc982629e0a"}
Jul 16 22:53:57.202864 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:57.202814    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" podStartSLOduration=16.202791077 podStartE2EDuration="16.202791077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:53:57.200642398 +0000 UTC m=+2193.521198471" watchObservedRunningTime="2026-07-16 22:53:57.202791077 +0000 UTC m=+2193.523347183"
Jul 16 22:53:58.189183 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:58.189128    2564 generic.go:356] "Generic (PLEG): container finished" podID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerID="1c51c139dd9db2b8e69a3b305aad910128b9f958a55e6a93ec4ccbc982629e0a" exitCode=6
Jul 16 22:53:58.189629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:58.189201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" event={"ID":"828544e4-dd4a-464b-b6ba-c521ab4d5217","Type":"ContainerDied","Data":"1c51c139dd9db2b8e69a3b305aad910128b9f958a55e6a93ec4ccbc982629e0a"}
Jul 16 22:53:58.189629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:58.189259    2564 scope.go:117] "RemoveContainer" containerID="3a23c761f65afbdc638cb538f3cd7ebbe1ee9f48bc94f3600bffd296f21ef072"
Jul 16 22:53:58.189629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:58.189499    2564 scope.go:117] "RemoveContainer" containerID="1c51c139dd9db2b8e69a3b305aad910128b9f958a55e6a93ec4ccbc982629e0a"
Jul 16 22:53:58.189745 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:53:58.189718    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt_odh-ai-gateway-infra(828544e4-dd4a-464b-b6ba-c521ab4d5217)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217"
Jul 16 22:53:58.214958 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:58.214932    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"]
Jul 16 22:53:59.330556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.330533    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:53:59.412799 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.412760    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828544e4-dd4a-464b-b6ba-c521ab4d5217-maas-api-service-ca\") pod \"828544e4-dd4a-464b-b6ba-c521ab4d5217\" (UID: \"828544e4-dd4a-464b-b6ba-c521ab4d5217\") "
Jul 16 22:53:59.413155 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.413110    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/828544e4-dd4a-464b-b6ba-c521ab4d5217-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "828544e4-dd4a-464b-b6ba-c521ab4d5217" (UID: "828544e4-dd4a-464b-b6ba-c521ab4d5217"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:53:59.513735 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.513629    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/828544e4-dd4a-464b-b6ba-c521ab4d5217-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:53:59.907701 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.907605    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"]
Jul 16 22:53:59.907997 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.907981    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.908044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.907998    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.908044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908007    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.908044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908013    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.908044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908037    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy"
Jul 16 22:53:59.908044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908044    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908052    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908059    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908111    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908119    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908128    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06da0dba-b829-4539-a577-c43a8a3f336d" containerName="istio-proxy"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908148    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.908229 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.908156    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:53:59.911117 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.911099    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:53:59.921742 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:53:59.921717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"]
Jul 16 22:54:00.017499 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.017460    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99c5fde4-2037-4960-b78e-9e3bb1204ea8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh\" (UID: \"99c5fde4-2037-4960-b78e-9e3bb1204ea8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:54:00.118926 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.118881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99c5fde4-2037-4960-b78e-9e3bb1204ea8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh\" (UID: \"99c5fde4-2037-4960-b78e-9e3bb1204ea8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:54:00.119565 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.119545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99c5fde4-2037-4960-b78e-9e3bb1204ea8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh\" (UID: \"99c5fde4-2037-4960-b78e-9e3bb1204ea8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:54:00.201924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.201896    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"
Jul 16 22:54:00.202108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.201926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt" event={"ID":"828544e4-dd4a-464b-b6ba-c521ab4d5217","Type":"ContainerDied","Data":"43d8dad2784ae01935a4e044666ed18a68d3f4617b23debcd65603ede2e251a9"}
Jul 16 22:54:00.202108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.201981    2564 scope.go:117] "RemoveContainer" containerID="1c51c139dd9db2b8e69a3b305aad910128b9f958a55e6a93ec4ccbc982629e0a"
Jul 16 22:54:00.221637 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.221613    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:54:00.223106 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.223087    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"]
Jul 16 22:54:00.226890 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.226867    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gpcgt"]
Jul 16 22:54:00.360910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:00.360886    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"]
Jul 16 22:54:00.362845 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:54:00.362814    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c5fde4_2037_4960_b78e_9e3bb1204ea8.slice/crio-8803edacca568b0f51987d63b0fde5813fc369f1985ee4d783a797ba03b4caf8 WatchSource:0}: Error finding container 8803edacca568b0f51987d63b0fde5813fc369f1985ee4d783a797ba03b4caf8: Status 404 returned error can't find the container with id 8803edacca568b0f51987d63b0fde5813fc369f1985ee4d783a797ba03b4caf8
Jul 16 22:54:01.040992 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.040954    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"]
Jul 16 22:54:01.041328 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.041312    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:54:01.041380 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.041330    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" containerName="revoke-keys"
Jul 16 22:54:01.044295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.044278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.046642 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.046620    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-75e8dc20-openshift-default-dockercfg-jnj64\""
Jul 16 22:54:01.063801 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.063776    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"]
Jul 16 22:54:01.129256 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-data\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129256 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-istio-token\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129256 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z96cg\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-kube-api-access-z96cg\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-socket\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e082596-ea6e-472b-84d5-1412f4897d74-istiod-ca-cert\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-credential-socket\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129360    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-envoy\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e082596-ea6e-472b-84d5-1412f4897d74-istio-podinfo\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.129547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.129442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-certs\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.208187 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.208127    2564 generic.go:356] "Generic (PLEG): container finished" podID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerID="e276b668e9f390eb181c4b2814679bbd894636ee88e6e3e5c4b779f626a09614" exitCode=6
Jul 16 22:54:01.208365 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.208237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" event={"ID":"99c5fde4-2037-4960-b78e-9e3bb1204ea8","Type":"ContainerDied","Data":"e276b668e9f390eb181c4b2814679bbd894636ee88e6e3e5c4b779f626a09614"}
Jul 16 22:54:01.208365 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.208328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" event={"ID":"99c5fde4-2037-4960-b78e-9e3bb1204ea8","Type":"ContainerStarted","Data":"8803edacca568b0f51987d63b0fde5813fc369f1985ee4d783a797ba03b4caf8"}
Jul 16 22:54:01.208464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.208431    2564 scope.go:117] "RemoveContainer" containerID="e276b668e9f390eb181c4b2814679bbd894636ee88e6e3e5c4b779f626a09614"
Jul 16 22:54:01.230520 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-data\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-istio-token\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z96cg\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-kube-api-access-z96cg\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-socket\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e082596-ea6e-472b-84d5-1412f4897d74-istiod-ca-cert\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-credential-socket\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-envoy\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230967 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e082596-ea6e-472b-84d5-1412f4897d74-istio-podinfo\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230967 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-certs\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230967 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230934    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-data\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.230967 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.230960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-socket\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.231197 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.231090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-credential-socket\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.231197 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.231154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-certs\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.231313 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.231294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e082596-ea6e-472b-84d5-1412f4897d74-istiod-ca-cert\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.232945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.232925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-envoy\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.233173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.233156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e082596-ea6e-472b-84d5-1412f4897d74-istio-podinfo\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.240910 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.240892    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-istio-token\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.240979 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.240961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z96cg\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-kube-api-access-z96cg\") pod \"e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") " pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.354888 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.354851    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:01.571675 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.571647    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"]
Jul 16 22:54:01.573806 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:54:01.573778    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e082596_ea6e_472b_84d5_1412f4897d74.slice/crio-e212f850a0ec86a81d8a7255f30f3c449d6f4eed7d30c63a367a5cfaba8a4e20 WatchSource:0}: Error finding container e212f850a0ec86a81d8a7255f30f3c449d6f4eed7d30c63a367a5cfaba8a4e20: Status 404 returned error can't find the container with id e212f850a0ec86a81d8a7255f30f3c449d6f4eed7d30c63a367a5cfaba8a4e20
Jul 16 22:54:01.576094 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.576062    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:54:01.576184 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.576146    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:54:01.576184 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.576177    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:54:01.602364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:01.602335    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828544e4-dd4a-464b-b6ba-c521ab4d5217" path="/var/lib/kubelet/pods/828544e4-dd4a-464b-b6ba-c521ab4d5217/volumes"
Jul 16 22:54:02.216751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.216714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" event={"ID":"6e082596-ea6e-472b-84d5-1412f4897d74","Type":"ContainerStarted","Data":"8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"}
Jul 16 22:54:02.216923 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.216772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" event={"ID":"6e082596-ea6e-472b-84d5-1412f4897d74","Type":"ContainerStarted","Data":"e212f850a0ec86a81d8a7255f30f3c449d6f4eed7d30c63a367a5cfaba8a4e20"}
Jul 16 22:54:02.218513 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.218482    2564 generic.go:356] "Generic (PLEG): container finished" podID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerID="eec33be76913d234c8202bbe80f3a1c3ee99d78a156e145e1d2357e59fc3b881" exitCode=6
Jul 16 22:54:02.218633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.218566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" event={"ID":"99c5fde4-2037-4960-b78e-9e3bb1204ea8","Type":"ContainerDied","Data":"eec33be76913d234c8202bbe80f3a1c3ee99d78a156e145e1d2357e59fc3b881"}
Jul 16 22:54:02.218633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.218624    2564 scope.go:117] "RemoveContainer" containerID="e276b668e9f390eb181c4b2814679bbd894636ee88e6e3e5c4b779f626a09614"
Jul 16 22:54:02.218766 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.218748    2564 scope.go:117] "RemoveContainer" containerID="eec33be76913d234c8202bbe80f3a1c3ee99d78a156e145e1d2357e59fc3b881"
Jul 16 22:54:02.219008 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:02.218990    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh_odh-ai-gateway-infra(99c5fde4-2037-4960-b78e-9e3bb1204ea8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8"
Jul 16 22:54:02.253314 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.253267    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podStartSLOduration=1.253249826 podStartE2EDuration="1.253249826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:54:02.252275311 +0000 UTC m=+2198.572831383" watchObservedRunningTime="2026-07-16 22:54:02.253249826 +0000 UTC m=+2198.573805900"
Jul 16 22:54:02.355262 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.355233    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:02.360284 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:02.360256    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:03.228824 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:03.228798    2564 scope.go:117] "RemoveContainer" containerID="eec33be76913d234c8202bbe80f3a1c3ee99d78a156e145e1d2357e59fc3b881"
Jul 16 22:54:03.229286 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:03.229008    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:03.229439 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:03.229048    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh_odh-ai-gateway-infra(99c5fde4-2037-4960-b78e-9e3bb1204ea8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8"
Jul 16 22:54:03.230000 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:03.229979    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:04.902272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:04.902241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:54:06.596805 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.596775    2564 scope.go:117] "RemoveContainer" containerID="8c84af0f99ca344aa753112a088dc1cb84298d102ec24049a803dcf55d2e037b"
Jul 16 22:54:06.603795 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.603769    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"]
Jul 16 22:54:06.608587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.608568    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:06.616333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.616304    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"]
Jul 16 22:54:06.684160 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.682713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ab73a4-f2bd-49f3-92c4-15052ea7634f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd\" (UID: \"e4ab73a4-f2bd-49f3-92c4-15052ea7634f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:06.784821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.783860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ab73a4-f2bd-49f3-92c4-15052ea7634f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd\" (UID: \"e4ab73a4-f2bd-49f3-92c4-15052ea7634f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:06.784821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.784763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ab73a4-f2bd-49f3-92c4-15052ea7634f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd\" (UID: \"e4ab73a4-f2bd-49f3-92c4-15052ea7634f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:06.933914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:06.933833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:07.106478 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:07.106454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"]
Jul 16 22:54:07.108129 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:54:07.108100    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ab73a4_f2bd_49f3_92c4_15052ea7634f.slice/crio-a974673b43feb96497051bf995c0977ab7e81cdc53ccf50fb3bcd542d13b9010 WatchSource:0}: Error finding container a974673b43feb96497051bf995c0977ab7e81cdc53ccf50fb3bcd542d13b9010: Status 404 returned error can't find the container with id a974673b43feb96497051bf995c0977ab7e81cdc53ccf50fb3bcd542d13b9010
Jul 16 22:54:07.252195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:07.252162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerStarted","Data":"dc1eadf18b9e505d55debea3d3da918b256cf5652c40792ea23b27f715ae0cd5"}
Jul 16 22:54:07.253498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:07.253472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" event={"ID":"e4ab73a4-f2bd-49f3-92c4-15052ea7634f","Type":"ContainerStarted","Data":"a974673b43feb96497051bf995c0977ab7e81cdc53ccf50fb3bcd542d13b9010"}
Jul 16 22:54:08.261254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.261160    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerID="21702c13861804df9d04c78843253c8bc658f926366dcc13b2c5d3d2e9d6281a" exitCode=7
Jul 16 22:54:08.261254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.261228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" event={"ID":"e4ab73a4-f2bd-49f3-92c4-15052ea7634f","Type":"ContainerDied","Data":"21702c13861804df9d04c78843253c8bc658f926366dcc13b2c5d3d2e9d6281a"}
Jul 16 22:54:08.261741 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.261446    2564 scope.go:117] "RemoveContainer" containerID="21702c13861804df9d04c78843253c8bc658f926366dcc13b2c5d3d2e9d6281a"
Jul 16 22:54:08.263361 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.263340    2564 generic.go:356] "Generic (PLEG): container finished" podID="25b41c74-465f-4483-8319-d719d40a317a" containerID="dc1eadf18b9e505d55debea3d3da918b256cf5652c40792ea23b27f715ae0cd5" exitCode=6
Jul 16 22:54:08.263498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.263401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerDied","Data":"dc1eadf18b9e505d55debea3d3da918b256cf5652c40792ea23b27f715ae0cd5"}
Jul 16 22:54:08.263498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.263447    2564 scope.go:117] "RemoveContainer" containerID="8c84af0f99ca344aa753112a088dc1cb84298d102ec24049a803dcf55d2e037b"
Jul 16 22:54:08.263703 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.263687    2564 scope.go:117] "RemoveContainer" containerID="dc1eadf18b9e505d55debea3d3da918b256cf5652c40792ea23b27f715ae0cd5"
Jul 16 22:54:08.263911 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:08.263894    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-z87s6_odh-ai-gateway-infra(25b41c74-465f-4483-8319-d719d40a317a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" podUID="25b41c74-465f-4483-8319-d719d40a317a"
Jul 16 22:54:08.316749 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:08.316710    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"]
Jul 16 22:54:09.271655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:09.271621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" event={"ID":"e4ab73a4-f2bd-49f3-92c4-15052ea7634f","Type":"ContainerStarted","Data":"787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec"}
Jul 16 22:54:09.296160 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:09.293901    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" podStartSLOduration=3.293878719 podStartE2EDuration="3.293878719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:54:09.291595506 +0000 UTC m=+2205.612151579" watchObservedRunningTime="2026-07-16 22:54:09.293878719 +0000 UTC m=+2205.614434788"
Jul 16 22:54:09.444273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:09.444249    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:54:09.613164 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:09.613061    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b41c74-465f-4483-8319-d719d40a317a-maas-api-service-ca\") pod \"25b41c74-465f-4483-8319-d719d40a317a\" (UID: \"25b41c74-465f-4483-8319-d719d40a317a\") "
Jul 16 22:54:09.613484 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:09.613454    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b41c74-465f-4483-8319-d719d40a317a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "25b41c74-465f-4483-8319-d719d40a317a" (UID: "25b41c74-465f-4483-8319-d719d40a317a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:54:09.714714 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:09.714676    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b41c74-465f-4483-8319-d719d40a317a-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:54:10.285492 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.285447    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerID="787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec" exitCode=7
Jul 16 22:54:10.286183 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.286156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" event={"ID":"e4ab73a4-f2bd-49f3-92c4-15052ea7634f","Type":"ContainerDied","Data":"787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec"}
Jul 16 22:54:10.286419 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.286388    2564 scope.go:117] "RemoveContainer" containerID="21702c13861804df9d04c78843253c8bc658f926366dcc13b2c5d3d2e9d6281a"
Jul 16 22:54:10.287055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.287038    2564 scope.go:117] "RemoveContainer" containerID="787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec"
Jul 16 22:54:10.287568 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:10.287544    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd_odh-ai-gateway-infra(e4ab73a4-f2bd-49f3-92c4-15052ea7634f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f"
Jul 16 22:54:10.293165 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.292386    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6" event={"ID":"25b41c74-465f-4483-8319-d719d40a317a","Type":"ContainerDied","Data":"a431514f556254536e2721948fc34647dac2d0f92f4d2a7ee4eb9acde3e26af5"}
Jul 16 22:54:10.293165 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.292483    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"
Jul 16 22:54:10.302923 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.302902    2564 scope.go:117] "RemoveContainer" containerID="dc1eadf18b9e505d55debea3d3da918b256cf5652c40792ea23b27f715ae0cd5"
Jul 16 22:54:10.327843 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.327817    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"]
Jul 16 22:54:10.330338 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:10.330314    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-z87s6"]
Jul 16 22:54:11.299872 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:11.299842    2564 scope.go:117] "RemoveContainer" containerID="787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec"
Jul 16 22:54:11.300338 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:11.300119    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd_odh-ai-gateway-infra(e4ab73a4-f2bd-49f3-92c4-15052ea7634f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f"
Jul 16 22:54:11.603545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:11.603444    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25b41c74-465f-4483-8319-d719d40a317a" path="/var/lib/kubelet/pods/25b41c74-465f-4483-8319-d719d40a317a/volumes"
Jul 16 22:54:13.313189 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.313155    2564 generic.go:356] "Generic (PLEG): container finished" podID="6e082596-ea6e-472b-84d5-1412f4897d74" containerID="8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114" exitCode=0
Jul 16 22:54:13.313690 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.313230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" event={"ID":"6e082596-ea6e-472b-84d5-1412f4897d74","Type":"ContainerDied","Data":"8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"}
Jul 16 22:54:13.313690 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.313597    2564 scope.go:117] "RemoveContainer" containerID="8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"
Jul 16 22:54:13.314673 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.314642    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:54:13.314769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.314710    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:54:13.314769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.314736    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:54:13.596057 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:13.595987    2564 scope.go:117] "RemoveContainer" containerID="eec33be76913d234c8202bbe80f3a1c3ee99d78a156e145e1d2357e59fc3b881"
Jul 16 22:54:14.320754 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.320720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" event={"ID":"6e082596-ea6e-472b-84d5-1412f4897d74","Type":"ContainerStarted","Data":"879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795"}
Jul 16 22:54:14.322648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.322623    2564 generic.go:356] "Generic (PLEG): container finished" podID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerID="086db1c68cc7d3866242c4dee609115052d7a46c63e90a68258cea6c2ab72101" exitCode=6
Jul 16 22:54:14.322771 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.322680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" event={"ID":"99c5fde4-2037-4960-b78e-9e3bb1204ea8","Type":"ContainerDied","Data":"086db1c68cc7d3866242c4dee609115052d7a46c63e90a68258cea6c2ab72101"}
Jul 16 22:54:14.322771 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.322730    2564 scope.go:117] "RemoveContainer" containerID="eec33be76913d234c8202bbe80f3a1c3ee99d78a156e145e1d2357e59fc3b881"
Jul 16 22:54:14.322980 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.322960    2564 scope.go:117] "RemoveContainer" containerID="086db1c68cc7d3866242c4dee609115052d7a46c63e90a68258cea6c2ab72101"
Jul 16 22:54:14.323192 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:14.323173    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh_odh-ai-gateway-infra(99c5fde4-2037-4960-b78e-9e3bb1204ea8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8"
Jul 16 22:54:14.355626 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.355596    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:14.356677 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.356651    2564 patch_prober.go:28] interesting pod/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 16 22:54:14.356792 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:14.356707    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 16 22:54:15.356158 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.356055    2564 patch_prober.go:28] interesting pod/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 16 22:54:15.356158 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.356109    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 16 22:54:15.378409 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.378372    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"]
Jul 16 22:54:15.547074 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.547050    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:54:15.693518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.693421    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99c5fde4-2037-4960-b78e-9e3bb1204ea8-maas-api-service-ca\") pod \"99c5fde4-2037-4960-b78e-9e3bb1204ea8\" (UID: \"99c5fde4-2037-4960-b78e-9e3bb1204ea8\") "
Jul 16 22:54:15.693789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.693763    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99c5fde4-2037-4960-b78e-9e3bb1204ea8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99c5fde4-2037-4960-b78e-9e3bb1204ea8" (UID: "99c5fde4-2037-4960-b78e-9e3bb1204ea8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:54:15.693912 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:15.693888    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99c5fde4-2037-4960-b78e-9e3bb1204ea8-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:54:16.338663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.338630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh" event={"ID":"99c5fde4-2037-4960-b78e-9e3bb1204ea8","Type":"ContainerDied","Data":"8803edacca568b0f51987d63b0fde5813fc369f1985ee4d783a797ba03b4caf8"}
Jul 16 22:54:16.338663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.338657    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"
Jul 16 22:54:16.338950 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.338689    2564 scope.go:117] "RemoveContainer" containerID="086db1c68cc7d3866242c4dee609115052d7a46c63e90a68258cea6c2ab72101"
Jul 16 22:54:16.356087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.356067    2564 patch_prober.go:28] interesting pod/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 16 22:54:16.356201 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.356120    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 16 22:54:16.360245 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.360219    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"]
Jul 16 22:54:16.364921 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:16.364894    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mwnmh"]
Jul 16 22:54:17.356018 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:17.355987    2564 patch_prober.go:28] interesting pod/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 16 22:54:17.356200 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:17.356035    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 16 22:54:17.601996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:17.601965    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" path="/var/lib/kubelet/pods/99c5fde4-2037-4960-b78e-9e3bb1204ea8/volumes"
Jul 16 22:54:18.228976 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:18.228938    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:18.355527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:18.355495    2564 patch_prober.go:28] interesting pod/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Jul 16 22:54:18.355703 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:18.355545    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Jul 16 22:54:19.359303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:19.359273    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:19.360450 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:19.360431    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:54:25.599116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:25.599083    2564 scope.go:117] "RemoveContainer" containerID="787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec"
Jul 16 22:54:26.395034 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:26.394997    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerID="b311dfdc5e72a9ab623d17312284cc582fe3e6bd8c2a141077ca19dbed8e3b28" exitCode=7
Jul 16 22:54:26.395220 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:26.395071    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" event={"ID":"e4ab73a4-f2bd-49f3-92c4-15052ea7634f","Type":"ContainerDied","Data":"b311dfdc5e72a9ab623d17312284cc582fe3e6bd8c2a141077ca19dbed8e3b28"}
Jul 16 22:54:26.395220 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:26.395193    2564 scope.go:117] "RemoveContainer" containerID="787d08bce7eb74e95ead20c4e67fb69de1c626c41e7f04ed6a05de7a38176bec"
Jul 16 22:54:26.395545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:26.395524    2564 scope.go:117] "RemoveContainer" containerID="b311dfdc5e72a9ab623d17312284cc582fe3e6bd8c2a141077ca19dbed8e3b28"
Jul 16 22:54:26.395772 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:26.395752    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd_odh-ai-gateway-infra(e4ab73a4-f2bd-49f3-92c4-15052ea7634f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f"
Jul 16 22:54:27.422437 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:27.422404    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"]
Jul 16 22:54:27.548536 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:27.548509    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:27.597115 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:27.597082    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ab73a4-f2bd-49f3-92c4-15052ea7634f-maas-api-service-ca\") pod \"e4ab73a4-f2bd-49f3-92c4-15052ea7634f\" (UID: \"e4ab73a4-f2bd-49f3-92c4-15052ea7634f\") "
Jul 16 22:54:27.597462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:27.597441    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ab73a4-f2bd-49f3-92c4-15052ea7634f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4ab73a4-f2bd-49f3-92c4-15052ea7634f" (UID: "e4ab73a4-f2bd-49f3-92c4-15052ea7634f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:54:27.698842 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:27.698810    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ab73a4-f2bd-49f3-92c4-15052ea7634f-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:54:28.407691 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:28.407661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"
Jul 16 22:54:28.407691 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:28.407673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd" event={"ID":"e4ab73a4-f2bd-49f3-92c4-15052ea7634f","Type":"ContainerDied","Data":"a974673b43feb96497051bf995c0977ab7e81cdc53ccf50fb3bcd542d13b9010"}
Jul 16 22:54:28.407938 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:28.407728    2564 scope.go:117] "RemoveContainer" containerID="b311dfdc5e72a9ab623d17312284cc582fe3e6bd8c2a141077ca19dbed8e3b28"
Jul 16 22:54:28.430124 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:28.430101    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"]
Jul 16 22:54:28.433264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:28.433242    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-7rkkd"]
Jul 16 22:54:29.601676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:29.601640    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" path="/var/lib/kubelet/pods/e4ab73a4-f2bd-49f3-92c4-15052ea7634f/volumes"
Jul 16 22:54:31.960170 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960120    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"]
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960464    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960475    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960492    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960498    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960505    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960511    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960518    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960523    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960529    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960533    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960543    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960549    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960556    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960553 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960561    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960568    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960573    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960623    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960632    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960638    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960645    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960650    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:31.960983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.960656    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:31.963573 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.963556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:31.965895 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.965877    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:54:31.965978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.965878    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:54:31.972626 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:31.972605    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"]
Jul 16 22:54:32.035365 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:32.035327    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4cd5c5c-debf-408f-a577-9963421fd9b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr\" (UID: \"a4cd5c5c-debf-408f-a577-9963421fd9b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:32.136421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:32.136382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4cd5c5c-debf-408f-a577-9963421fd9b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr\" (UID: \"a4cd5c5c-debf-408f-a577-9963421fd9b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:32.137047 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:32.137027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4cd5c5c-debf-408f-a577-9963421fd9b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr\" (UID: \"a4cd5c5c-debf-408f-a577-9963421fd9b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:32.274970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:32.274859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:32.406286 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:32.406260    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"]
Jul 16 22:54:32.407173 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:54:32.407128    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4cd5c5c_debf_408f_a577_9963421fd9b5.slice/crio-58e34b5bc7eafd858929714f56fa3483b9183d69681414202ec73db2f011e686 WatchSource:0}: Error finding container 58e34b5bc7eafd858929714f56fa3483b9183d69681414202ec73db2f011e686: Status 404 returned error can't find the container with id 58e34b5bc7eafd858929714f56fa3483b9183d69681414202ec73db2f011e686
Jul 16 22:54:32.434554 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:32.434527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" event={"ID":"a4cd5c5c-debf-408f-a577-9963421fd9b5","Type":"ContainerStarted","Data":"58e34b5bc7eafd858929714f56fa3483b9183d69681414202ec73db2f011e686"}
Jul 16 22:54:33.441000 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:33.440912    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerID="31b8b771465734a9413180bbc03a57e2d503d5bd02c8a8943784644f22af2376" exitCode=6
Jul 16 22:54:33.441376 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:33.441004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" event={"ID":"a4cd5c5c-debf-408f-a577-9963421fd9b5","Type":"ContainerDied","Data":"31b8b771465734a9413180bbc03a57e2d503d5bd02c8a8943784644f22af2376"}
Jul 16 22:54:33.441376 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:33.441161    2564 scope.go:117] "RemoveContainer" containerID="31b8b771465734a9413180bbc03a57e2d503d5bd02c8a8943784644f22af2376"
Jul 16 22:54:34.448043 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:34.448009    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerID="dc793c8f9c23be410abf8db01a56fabbf986e7bfb0feee038d08cfc105350a30" exitCode=6
Jul 16 22:54:34.448492 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:34.448094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" event={"ID":"a4cd5c5c-debf-408f-a577-9963421fd9b5","Type":"ContainerDied","Data":"dc793c8f9c23be410abf8db01a56fabbf986e7bfb0feee038d08cfc105350a30"}
Jul 16 22:54:34.448492 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:34.448195    2564 scope.go:117] "RemoveContainer" containerID="31b8b771465734a9413180bbc03a57e2d503d5bd02c8a8943784644f22af2376"
Jul 16 22:54:34.448492 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:34.448468    2564 scope.go:117] "RemoveContainer" containerID="dc793c8f9c23be410abf8db01a56fabbf986e7bfb0feee038d08cfc105350a30"
Jul 16 22:54:34.448739 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:34.448716    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr_odh-ai-gateway-infra(a4cd5c5c-debf-408f-a577-9963421fd9b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5"
Jul 16 22:54:35.455937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:35.455908    2564 scope.go:117] "RemoveContainer" containerID="dc793c8f9c23be410abf8db01a56fabbf986e7bfb0feee038d08cfc105350a30"
Jul 16 22:54:35.456350 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:35.456242    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr_odh-ai-gateway-infra(a4cd5c5c-debf-408f-a577-9963421fd9b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5"
Jul 16 22:54:46.596883 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:46.596802    2564 scope.go:117] "RemoveContainer" containerID="dc793c8f9c23be410abf8db01a56fabbf986e7bfb0feee038d08cfc105350a30"
Jul 16 22:54:47.517518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:47.517481    2564 generic.go:356] "Generic (PLEG): container finished" podID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerID="b17ae42e57b8d43ad4f6ff62f9983c497ff4db49ffbf44c921ded56f59becb07" exitCode=6
Jul 16 22:54:47.517703 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:47.517554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" event={"ID":"a4cd5c5c-debf-408f-a577-9963421fd9b5","Type":"ContainerDied","Data":"b17ae42e57b8d43ad4f6ff62f9983c497ff4db49ffbf44c921ded56f59becb07"}
Jul 16 22:54:47.517703 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:47.517612    2564 scope.go:117] "RemoveContainer" containerID="dc793c8f9c23be410abf8db01a56fabbf986e7bfb0feee038d08cfc105350a30"
Jul 16 22:54:47.517919 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:47.517903    2564 scope.go:117] "RemoveContainer" containerID="b17ae42e57b8d43ad4f6ff62f9983c497ff4db49ffbf44c921ded56f59becb07"
Jul 16 22:54:47.518193 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:47.518169    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr_odh-ai-gateway-infra(a4cd5c5c-debf-408f-a577-9963421fd9b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5"
Jul 16 22:54:48.508294 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.508262    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"]
Jul 16 22:54:48.508665 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.508622    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:48.508665 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.508634    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:48.508737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.508705    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="25b41c74-465f-4483-8319-d719d40a317a" containerName="revoke-keys"
Jul 16 22:54:48.508737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.508713    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ab73a4-f2bd-49f3-92c4-15052ea7634f" containerName="revoke-keys"
Jul 16 22:54:48.508737 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.508721    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c5fde4-2037-4960-b78e-9e3bb1204ea8" containerName="revoke-keys"
Jul 16 22:54:48.511428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.511412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:54:48.523269 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.523234    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"]
Jul 16 22:54:48.548945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.548915    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"]
Jul 16 22:54:48.589280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.589249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be32365-2236-4edf-8ea6-25b705699a4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-4d26x\" (UID: \"5be32365-2236-4edf-8ea6-25b705699a4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:54:48.681089 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.681065    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:48.690111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.690086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be32365-2236-4edf-8ea6-25b705699a4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-4d26x\" (UID: \"5be32365-2236-4edf-8ea6-25b705699a4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:54:48.690667 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.690650    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be32365-2236-4edf-8ea6-25b705699a4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-4d26x\" (UID: \"5be32365-2236-4edf-8ea6-25b705699a4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:54:48.790991 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.790892    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4cd5c5c-debf-408f-a577-9963421fd9b5-maas-api-service-ca\") pod \"a4cd5c5c-debf-408f-a577-9963421fd9b5\" (UID: \"a4cd5c5c-debf-408f-a577-9963421fd9b5\") "
Jul 16 22:54:48.791304 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.791281    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4cd5c5c-debf-408f-a577-9963421fd9b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4cd5c5c-debf-408f-a577-9963421fd9b5" (UID: "a4cd5c5c-debf-408f-a577-9963421fd9b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:54:48.822908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.822877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:54:48.891889 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.891857    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4cd5c5c-debf-408f-a577-9963421fd9b5-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:54:48.958023 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:48.957996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"]
Jul 16 22:54:48.960153 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:54:48.960103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be32365_2236_4edf_8ea6_25b705699a4a.slice/crio-a3a22a51517590599ce43ad2c9c3ce99c149041e3c877105a13c0f953cd7b7d1 WatchSource:0}: Error finding container a3a22a51517590599ce43ad2c9c3ce99c149041e3c877105a13c0f953cd7b7d1: Status 404 returned error can't find the container with id a3a22a51517590599ce43ad2c9c3ce99c149041e3c877105a13c0f953cd7b7d1
Jul 16 22:54:49.532945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.532919    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"
Jul 16 22:54:49.533454 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.532969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr" event={"ID":"a4cd5c5c-debf-408f-a577-9963421fd9b5","Type":"ContainerDied","Data":"58e34b5bc7eafd858929714f56fa3483b9183d69681414202ec73db2f011e686"}
Jul 16 22:54:49.533454 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.533030    2564 scope.go:117] "RemoveContainer" containerID="b17ae42e57b8d43ad4f6ff62f9983c497ff4db49ffbf44c921ded56f59becb07"
Jul 16 22:54:49.534708 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.534674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" event={"ID":"5be32365-2236-4edf-8ea6-25b705699a4a","Type":"ContainerStarted","Data":"41bcd55c63690abdcccc7a7c7a27bb24f93486ebdd98c30c11451bf9c3717ec6"}
Jul 16 22:54:49.534826 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.534722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" event={"ID":"5be32365-2236-4edf-8ea6-25b705699a4a","Type":"ContainerStarted","Data":"a3a22a51517590599ce43ad2c9c3ce99c149041e3c877105a13c0f953cd7b7d1"}
Jul 16 22:54:49.554631 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.554592    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" podStartSLOduration=1.554576996 podStartE2EDuration="1.554576996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:54:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:54:49.552778214 +0000 UTC m=+2245.873334277" watchObservedRunningTime="2026-07-16 22:54:49.554576996 +0000 UTC m=+2245.875133069"
Jul 16 22:54:49.578044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.578004    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"]
Jul 16 22:54:49.580408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.580384    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sdvwr"]
Jul 16 22:54:49.601899 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.601868    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" path="/var/lib/kubelet/pods/a4cd5c5c-debf-408f-a577-9963421fd9b5/volumes"
Jul 16 22:54:49.962233 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962201    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"]
Jul 16 22:54:49.962534 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962522    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.962574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962535    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.962574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962554    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.962574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962560    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.962686 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962617    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.962686 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962623    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.962686 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.962632    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:54:49.964750 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.964730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:54:49.975476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:49.975450    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"]
Jul 16 22:54:50.104970 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.104926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-t455g\" (UID: \"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:54:50.205453 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.205425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-t455g\" (UID: \"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:54:50.206270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.206246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-t455g\" (UID: \"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:54:50.276396 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.276292    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:54:50.410519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.410493    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"]
Jul 16 22:54:50.412309 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:54:50.412282    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed8fef3_9fee_4c1c_b320_ac8c1e33990a.slice/crio-d832874eec209fa270e8c9610ea5e4f6a66a2623fe37efcbfb879580623a840f WatchSource:0}: Error finding container d832874eec209fa270e8c9610ea5e4f6a66a2623fe37efcbfb879580623a840f: Status 404 returned error can't find the container with id d832874eec209fa270e8c9610ea5e4f6a66a2623fe37efcbfb879580623a840f
Jul 16 22:54:50.541618 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.541528    2564 generic.go:356] "Generic (PLEG): container finished" podID="5be32365-2236-4edf-8ea6-25b705699a4a" containerID="41bcd55c63690abdcccc7a7c7a27bb24f93486ebdd98c30c11451bf9c3717ec6" exitCode=6
Jul 16 22:54:50.542061 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.541618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" event={"ID":"5be32365-2236-4edf-8ea6-25b705699a4a","Type":"ContainerDied","Data":"41bcd55c63690abdcccc7a7c7a27bb24f93486ebdd98c30c11451bf9c3717ec6"}
Jul 16 22:54:50.542061 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.541849    2564 scope.go:117] "RemoveContainer" containerID="41bcd55c63690abdcccc7a7c7a27bb24f93486ebdd98c30c11451bf9c3717ec6"
Jul 16 22:54:50.543800 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:50.543775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" event={"ID":"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a","Type":"ContainerStarted","Data":"d832874eec209fa270e8c9610ea5e4f6a66a2623fe37efcbfb879580623a840f"}
Jul 16 22:54:51.549813 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.549783    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerID="d36ff937cebff96c05037109d0be50408c7c5271dfbf17de60c14da4ccc7d772" exitCode=6
Jul 16 22:54:51.550239 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.549853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" event={"ID":"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a","Type":"ContainerDied","Data":"d36ff937cebff96c05037109d0be50408c7c5271dfbf17de60c14da4ccc7d772"}
Jul 16 22:54:51.550239 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.550063    2564 scope.go:117] "RemoveContainer" containerID="d36ff937cebff96c05037109d0be50408c7c5271dfbf17de60c14da4ccc7d772"
Jul 16 22:54:51.551752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.551732    2564 generic.go:356] "Generic (PLEG): container finished" podID="5be32365-2236-4edf-8ea6-25b705699a4a" containerID="db8405d9947d6f833708430986e0b5e60eb22c4c1df74b4918316c6c8851250a" exitCode=6
Jul 16 22:54:51.551849 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.551824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" event={"ID":"5be32365-2236-4edf-8ea6-25b705699a4a","Type":"ContainerDied","Data":"db8405d9947d6f833708430986e0b5e60eb22c4c1df74b4918316c6c8851250a"}
Jul 16 22:54:51.551905 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.551886    2564 scope.go:117] "RemoveContainer" containerID="41bcd55c63690abdcccc7a7c7a27bb24f93486ebdd98c30c11451bf9c3717ec6"
Jul 16 22:54:51.552011 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:51.551996    2564 scope.go:117] "RemoveContainer" containerID="db8405d9947d6f833708430986e0b5e60eb22c4c1df74b4918316c6c8851250a"
Jul 16 22:54:51.552199 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:51.552170    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-4d26x_odh-ai-gateway-infra(5be32365-2236-4edf-8ea6-25b705699a4a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" podUID="5be32365-2236-4edf-8ea6-25b705699a4a"
Jul 16 22:54:52.559559 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:52.559465    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerID="72f52e1bb8a73af0b96aba5b50526c0148755e6d226a13bd9c3d40a975dc94ac" exitCode=6
Jul 16 22:54:52.559559 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:52.559546    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" event={"ID":"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a","Type":"ContainerDied","Data":"72f52e1bb8a73af0b96aba5b50526c0148755e6d226a13bd9c3d40a975dc94ac"}
Jul 16 22:54:52.560087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:52.559600    2564 scope.go:117] "RemoveContainer" containerID="d36ff937cebff96c05037109d0be50408c7c5271dfbf17de60c14da4ccc7d772"
Jul 16 22:54:52.560087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:52.559781    2564 scope.go:117] "RemoveContainer" containerID="72f52e1bb8a73af0b96aba5b50526c0148755e6d226a13bd9c3d40a975dc94ac"
Jul 16 22:54:52.560087 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:52.560047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-t455g_odh-ai-gateway-infra(7ed8fef3-9fee-4c1c-b320-ac8c1e33990a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a"
Jul 16 22:54:52.561389 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:52.561374    2564 scope.go:117] "RemoveContainer" containerID="db8405d9947d6f833708430986e0b5e60eb22c4c1df74b4918316c6c8851250a"
Jul 16 22:54:52.561550 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:52.561532    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-4d26x_odh-ai-gateway-infra(5be32365-2236-4edf-8ea6-25b705699a4a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" podUID="5be32365-2236-4edf-8ea6-25b705699a4a"
Jul 16 22:54:53.568372 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:54:53.568347    2564 scope.go:117] "RemoveContainer" containerID="72f52e1bb8a73af0b96aba5b50526c0148755e6d226a13bd9c3d40a975dc94ac"
Jul 16 22:54:53.568743 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:54:53.568543    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-t455g_odh-ai-gateway-infra(7ed8fef3-9fee-4c1c-b320-ac8c1e33990a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a"
Jul 16 22:55:03.595978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:03.595948    2564 scope.go:117] "RemoveContainer" containerID="db8405d9947d6f833708430986e0b5e60eb22c4c1df74b4918316c6c8851250a"
Jul 16 22:55:04.596851 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:04.596822    2564 scope.go:117] "RemoveContainer" containerID="72f52e1bb8a73af0b96aba5b50526c0148755e6d226a13bd9c3d40a975dc94ac"
Jul 16 22:55:04.632863 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:04.632835    2564 generic.go:356] "Generic (PLEG): container finished" podID="5be32365-2236-4edf-8ea6-25b705699a4a" containerID="f7b1dcc5d4af9c70372b158a9d0d2b61d132a25479b2e1101e48006ccf82d67e" exitCode=6
Jul 16 22:55:04.633009 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:04.632906    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" event={"ID":"5be32365-2236-4edf-8ea6-25b705699a4a","Type":"ContainerDied","Data":"f7b1dcc5d4af9c70372b158a9d0d2b61d132a25479b2e1101e48006ccf82d67e"}
Jul 16 22:55:04.633009 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:04.632963    2564 scope.go:117] "RemoveContainer" containerID="db8405d9947d6f833708430986e0b5e60eb22c4c1df74b4918316c6c8851250a"
Jul 16 22:55:04.633265 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:04.633247    2564 scope.go:117] "RemoveContainer" containerID="f7b1dcc5d4af9c70372b158a9d0d2b61d132a25479b2e1101e48006ccf82d67e"
Jul 16 22:55:04.633457 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:04.633440    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-4d26x_odh-ai-gateway-infra(5be32365-2236-4edf-8ea6-25b705699a4a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" podUID="5be32365-2236-4edf-8ea6-25b705699a4a"
Jul 16 22:55:05.639399 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.639365    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerID="90641c3ed783fce0170636c18eb5e24ff5ef9c3fc785c74abb4c04db82b1cf5b" exitCode=6
Jul 16 22:55:05.639876 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.639436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" event={"ID":"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a","Type":"ContainerDied","Data":"90641c3ed783fce0170636c18eb5e24ff5ef9c3fc785c74abb4c04db82b1cf5b"}
Jul 16 22:55:05.639876 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.639525    2564 scope.go:117] "RemoveContainer" containerID="72f52e1bb8a73af0b96aba5b50526c0148755e6d226a13bd9c3d40a975dc94ac"
Jul 16 22:55:05.639876 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.639865    2564 scope.go:117] "RemoveContainer" containerID="90641c3ed783fce0170636c18eb5e24ff5ef9c3fc785c74abb4c04db82b1cf5b"
Jul 16 22:55:05.640104 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:05.640081    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-t455g_odh-ai-gateway-infra(7ed8fef3-9fee-4c1c-b320-ac8c1e33990a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a"
Jul 16 22:55:05.667951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.667920    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"]
Jul 16 22:55:05.800179 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.800147    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:55:05.841408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.841361    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be32365-2236-4edf-8ea6-25b705699a4a-maas-api-service-ca\") pod \"5be32365-2236-4edf-8ea6-25b705699a4a\" (UID: \"5be32365-2236-4edf-8ea6-25b705699a4a\") "
Jul 16 22:55:05.841724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.841702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be32365-2236-4edf-8ea6-25b705699a4a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5be32365-2236-4edf-8ea6-25b705699a4a" (UID: "5be32365-2236-4edf-8ea6-25b705699a4a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:05.942297 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:05.942209    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be32365-2236-4edf-8ea6-25b705699a4a-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:55:06.649731 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.649702    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"
Jul 16 22:55:06.649731 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.649718    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x" event={"ID":"5be32365-2236-4edf-8ea6-25b705699a4a","Type":"ContainerDied","Data":"a3a22a51517590599ce43ad2c9c3ce99c149041e3c877105a13c0f953cd7b7d1"}
Jul 16 22:55:06.650251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.649781    2564 scope.go:117] "RemoveContainer" containerID="f7b1dcc5d4af9c70372b158a9d0d2b61d132a25479b2e1101e48006ccf82d67e"
Jul 16 22:55:06.660420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660397    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"]
Jul 16 22:55:06.660764 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660748    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:06.660840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660767    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:06.660840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660786    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:06.660840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660795    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:06.660840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660816    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:55:06.660840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660825    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4cd5c5c-debf-408f-a577-9963421fd9b5" containerName="revoke-keys"
Jul 16 22:55:06.661087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660918    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:06.661087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.660932    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:06.663476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.663458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:06.668924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.668904    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"]
Jul 16 22:55:06.674282 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.674258    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"]
Jul 16 22:55:06.677053 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.677027    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"]
Jul 16 22:55:06.684705 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.684680    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-4d26x"]
Jul 16 22:55:06.749768 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.749738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/968e9822-e1fa-4c2e-9a0c-7c558db809bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t\" (UID: \"968e9822-e1fa-4c2e-9a0c-7c558db809bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:06.808775 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.808749    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:55:06.850332 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.850302    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a-maas-api-service-ca\") pod \"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a\" (UID: \"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a\") "
Jul 16 22:55:06.850493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.850478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/968e9822-e1fa-4c2e-9a0c-7c558db809bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t\" (UID: \"968e9822-e1fa-4c2e-9a0c-7c558db809bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:06.850665 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.850640    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" (UID: "7ed8fef3-9fee-4c1c-b320-ac8c1e33990a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:06.851086 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.851070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/968e9822-e1fa-4c2e-9a0c-7c558db809bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t\" (UID: \"968e9822-e1fa-4c2e-9a0c-7c558db809bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:06.951575 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.951546    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:55:06.976456 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:06.976421    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:07.119787 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.119677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"]
Jul 16 22:55:07.122272 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:55:07.122248    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod968e9822_e1fa_4c2e_9a0c_7c558db809bb.slice/crio-dd4d54dcd74769a8434be8a3663ac28ecddfffce636af6bfd55a36a1a54add3a WatchSource:0}: Error finding container dd4d54dcd74769a8434be8a3663ac28ecddfffce636af6bfd55a36a1a54add3a: Status 404 returned error can't find the container with id dd4d54dcd74769a8434be8a3663ac28ecddfffce636af6bfd55a36a1a54add3a
Jul 16 22:55:07.602448 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.602361    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" path="/var/lib/kubelet/pods/5be32365-2236-4edf-8ea6-25b705699a4a/volumes"
Jul 16 22:55:07.655730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.655697    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"
Jul 16 22:55:07.656207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.655695    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g" event={"ID":"7ed8fef3-9fee-4c1c-b320-ac8c1e33990a","Type":"ContainerDied","Data":"d832874eec209fa270e8c9610ea5e4f6a66a2623fe37efcbfb879580623a840f"}
Jul 16 22:55:07.656207 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.655845    2564 scope.go:117] "RemoveContainer" containerID="90641c3ed783fce0170636c18eb5e24ff5ef9c3fc785c74abb4c04db82b1cf5b"
Jul 16 22:55:07.658695 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.658651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" event={"ID":"968e9822-e1fa-4c2e-9a0c-7c558db809bb","Type":"ContainerStarted","Data":"02a43740a1c823eca512e008e9e767c4e1235c5118d26a523ae6f71248441672"}
Jul 16 22:55:07.658832 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.658702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" event={"ID":"968e9822-e1fa-4c2e-9a0c-7c558db809bb","Type":"ContainerStarted","Data":"dd4d54dcd74769a8434be8a3663ac28ecddfffce636af6bfd55a36a1a54add3a"}
Jul 16 22:55:07.673680 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.673641    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"]
Jul 16 22:55:07.676202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.676180    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-t455g"]
Jul 16 22:55:07.692889 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:07.692841    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" podStartSLOduration=1.6928252590000001 podStartE2EDuration="1.692825259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:55:07.689784606 +0000 UTC m=+2264.010340676" watchObservedRunningTime="2026-07-16 22:55:07.692825259 +0000 UTC m=+2264.013381331"
Jul 16 22:55:08.672015 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:08.671981    2564 generic.go:356] "Generic (PLEG): container finished" podID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerID="02a43740a1c823eca512e008e9e767c4e1235c5118d26a523ae6f71248441672" exitCode=7
Jul 16 22:55:08.672490 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:08.672067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" event={"ID":"968e9822-e1fa-4c2e-9a0c-7c558db809bb","Type":"ContainerDied","Data":"02a43740a1c823eca512e008e9e767c4e1235c5118d26a523ae6f71248441672"}
Jul 16 22:55:08.672490 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:08.672258    2564 scope.go:117] "RemoveContainer" containerID="02a43740a1c823eca512e008e9e767c4e1235c5118d26a523ae6f71248441672"
Jul 16 22:55:09.601935 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:09.601899    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" path="/var/lib/kubelet/pods/7ed8fef3-9fee-4c1c-b320-ac8c1e33990a/volumes"
Jul 16 22:55:09.681102 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:09.681063    2564 generic.go:356] "Generic (PLEG): container finished" podID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerID="9e7970d8f64b87fc0b6ef76ac6ed4b5086ac4ddeb34f2efbaac96243bf5a84fc" exitCode=7
Jul 16 22:55:09.681594 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:09.681126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" event={"ID":"968e9822-e1fa-4c2e-9a0c-7c558db809bb","Type":"ContainerDied","Data":"9e7970d8f64b87fc0b6ef76ac6ed4b5086ac4ddeb34f2efbaac96243bf5a84fc"}
Jul 16 22:55:09.681594 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:09.681203    2564 scope.go:117] "RemoveContainer" containerID="02a43740a1c823eca512e008e9e767c4e1235c5118d26a523ae6f71248441672"
Jul 16 22:55:09.681594 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:09.681379    2564 scope.go:117] "RemoveContainer" containerID="9e7970d8f64b87fc0b6ef76ac6ed4b5086ac4ddeb34f2efbaac96243bf5a84fc"
Jul 16 22:55:09.681727 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:09.681626    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t_odh-ai-gateway-infra(968e9822-e1fa-4c2e-9a0c-7c558db809bb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb"
Jul 16 22:55:10.688241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:10.688216    2564 scope.go:117] "RemoveContainer" containerID="9e7970d8f64b87fc0b6ef76ac6ed4b5086ac4ddeb34f2efbaac96243bf5a84fc"
Jul 16 22:55:10.688612 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:10.688391    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t_odh-ai-gateway-infra(968e9822-e1fa-4c2e-9a0c-7c558db809bb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb"
Jul 16 22:55:21.595973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:21.595943    2564 scope.go:117] "RemoveContainer" containerID="9e7970d8f64b87fc0b6ef76ac6ed4b5086ac4ddeb34f2efbaac96243bf5a84fc"
Jul 16 22:55:22.013403 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013368    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"]
Jul 16 22:55:22.013731 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013710    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:22.013774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013732    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:22.013774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013745    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:22.013774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013750    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:22.013774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013764    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:22.013774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013769    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:22.013925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013829    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:22.013925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013838    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:22.013925 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.013845    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be32365-2236-4edf-8ea6-25b705699a4a" containerName="revoke-keys"
Jul 16 22:55:22.015774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.015756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:22.036487 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.036457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"]
Jul 16 22:55:22.073947 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.073915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6a31e8-4286-4fd6-86cd-e7af655220b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7\" (UID: \"1d6a31e8-4286-4fd6-86cd-e7af655220b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:22.174540 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.174516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6a31e8-4286-4fd6-86cd-e7af655220b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7\" (UID: \"1d6a31e8-4286-4fd6-86cd-e7af655220b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:22.175089 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.175069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6a31e8-4286-4fd6-86cd-e7af655220b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7\" (UID: \"1d6a31e8-4286-4fd6-86cd-e7af655220b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:22.325871 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.325840    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:22.466264 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.466235    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"]
Jul 16 22:55:22.467000 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:55:22.466976    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6a31e8_4286_4fd6_86cd_e7af655220b7.slice/crio-5104740ef907ead0c7de72340df45573cdb566cd850b2337bde3a523e8bb3a9d WatchSource:0}: Error finding container 5104740ef907ead0c7de72340df45573cdb566cd850b2337bde3a523e8bb3a9d: Status 404 returned error can't find the container with id 5104740ef907ead0c7de72340df45573cdb566cd850b2337bde3a523e8bb3a9d
Jul 16 22:55:22.750310 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.750210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" event={"ID":"1d6a31e8-4286-4fd6-86cd-e7af655220b7","Type":"ContainerStarted","Data":"ba76b44b5c7b9260690a166e594aa99777ead7ccb1e3e77f62c410c38c16a9da"}
Jul 16 22:55:22.750310 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.750273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" event={"ID":"1d6a31e8-4286-4fd6-86cd-e7af655220b7","Type":"ContainerStarted","Data":"5104740ef907ead0c7de72340df45573cdb566cd850b2337bde3a523e8bb3a9d"}
Jul 16 22:55:22.752026 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.752001    2564 generic.go:356] "Generic (PLEG): container finished" podID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerID="a1eee1f4611aad71e719e7a286f94343707941ee81bf3b686a1a1da841b387ff" exitCode=7
Jul 16 22:55:22.752169 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.752072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" event={"ID":"968e9822-e1fa-4c2e-9a0c-7c558db809bb","Type":"ContainerDied","Data":"a1eee1f4611aad71e719e7a286f94343707941ee81bf3b686a1a1da841b387ff"}
Jul 16 22:55:22.752169 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.752125    2564 scope.go:117] "RemoveContainer" containerID="9e7970d8f64b87fc0b6ef76ac6ed4b5086ac4ddeb34f2efbaac96243bf5a84fc"
Jul 16 22:55:22.752375 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.752360    2564 scope.go:117] "RemoveContainer" containerID="a1eee1f4611aad71e719e7a286f94343707941ee81bf3b686a1a1da841b387ff"
Jul 16 22:55:22.752587 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:22.752565    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t_odh-ai-gateway-infra(968e9822-e1fa-4c2e-9a0c-7c558db809bb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb"
Jul 16 22:55:22.770642 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:22.770584    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" podStartSLOduration=1.770563513 podStartE2EDuration="1.770563513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:55:22.767098169 +0000 UTC m=+2279.087654242" watchObservedRunningTime="2026-07-16 22:55:22.770563513 +0000 UTC m=+2279.091119586"
Jul 16 22:55:23.759126 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.759093    2564 generic.go:356] "Generic (PLEG): container finished" podID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerID="ba76b44b5c7b9260690a166e594aa99777ead7ccb1e3e77f62c410c38c16a9da" exitCode=6
Jul 16 22:55:23.759577 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.759176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" event={"ID":"1d6a31e8-4286-4fd6-86cd-e7af655220b7","Type":"ContainerDied","Data":"ba76b44b5c7b9260690a166e594aa99777ead7ccb1e3e77f62c410c38c16a9da"}
Jul 16 22:55:23.759577 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.759385    2564 scope.go:117] "RemoveContainer" containerID="ba76b44b5c7b9260690a166e594aa99777ead7ccb1e3e77f62c410c38c16a9da"
Jul 16 22:55:23.794429 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.794403    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"]
Jul 16 22:55:23.931011 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.930986    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:23.993173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.993120    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/968e9822-e1fa-4c2e-9a0c-7c558db809bb-maas-api-service-ca\") pod \"968e9822-e1fa-4c2e-9a0c-7c558db809bb\" (UID: \"968e9822-e1fa-4c2e-9a0c-7c558db809bb\") "
Jul 16 22:55:23.993472 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:23.993446    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/968e9822-e1fa-4c2e-9a0c-7c558db809bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "968e9822-e1fa-4c2e-9a0c-7c558db809bb" (UID: "968e9822-e1fa-4c2e-9a0c-7c558db809bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:24.093951 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.093927    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/968e9822-e1fa-4c2e-9a0c-7c558db809bb-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:55:24.767756 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.767720    2564 generic.go:356] "Generic (PLEG): container finished" podID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerID="c31e078627229f376df5e8f53af7a8b4d42ec63e6dee69fef7341d87252104fc" exitCode=6
Jul 16 22:55:24.768178 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.767802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" event={"ID":"1d6a31e8-4286-4fd6-86cd-e7af655220b7","Type":"ContainerDied","Data":"c31e078627229f376df5e8f53af7a8b4d42ec63e6dee69fef7341d87252104fc"}
Jul 16 22:55:24.768178 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.767889    2564 scope.go:117] "RemoveContainer" containerID="ba76b44b5c7b9260690a166e594aa99777ead7ccb1e3e77f62c410c38c16a9da"
Jul 16 22:55:24.768178 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.767996    2564 scope.go:117] "RemoveContainer" containerID="c31e078627229f376df5e8f53af7a8b4d42ec63e6dee69fef7341d87252104fc"
Jul 16 22:55:24.768356 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:24.768249    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7_odh-ai-gateway-infra(1d6a31e8-4286-4fd6-86cd-e7af655220b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7"
Jul 16 22:55:24.769156 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.769127    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"
Jul 16 22:55:24.769156 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.769129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t" event={"ID":"968e9822-e1fa-4c2e-9a0c-7c558db809bb","Type":"ContainerDied","Data":"dd4d54dcd74769a8434be8a3663ac28ecddfffce636af6bfd55a36a1a54add3a"}
Jul 16 22:55:24.780794 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.780713    2564 scope.go:117] "RemoveContainer" containerID="a1eee1f4611aad71e719e7a286f94343707941ee81bf3b686a1a1da841b387ff"
Jul 16 22:55:24.799175 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.799146    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"]
Jul 16 22:55:24.809648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:24.809622    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-8mc9t"]
Jul 16 22:55:25.601749 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:25.601719    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" path="/var/lib/kubelet/pods/968e9822-e1fa-4c2e-9a0c-7c558db809bb/volumes"
Jul 16 22:55:25.775751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:25.775722    2564 scope.go:117] "RemoveContainer" containerID="c31e078627229f376df5e8f53af7a8b4d42ec63e6dee69fef7341d87252104fc"
Jul 16 22:55:25.776174 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:25.775980    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7_odh-ai-gateway-infra(1d6a31e8-4286-4fd6-86cd-e7af655220b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7"
Jul 16 22:55:38.552314 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552275    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"]
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552621    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552631    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552646    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552652    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552663    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552670    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552726    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552737    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed8fef3-9fee-4c1c-b320-ac8c1e33990a" containerName="revoke-keys"
Jul 16 22:55:38.552778 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.552744    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:38.555744 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.555728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:38.563964 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.563935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"]
Jul 16 22:55:38.624813 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.624775    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27dc5a33-11f9-441d-9b56-0e6b33e3cd49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-g8nq8\" (UID: \"27dc5a33-11f9-441d-9b56-0e6b33e3cd49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:38.725519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.725478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27dc5a33-11f9-441d-9b56-0e6b33e3cd49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-g8nq8\" (UID: \"27dc5a33-11f9-441d-9b56-0e6b33e3cd49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:38.726127 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.726106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27dc5a33-11f9-441d-9b56-0e6b33e3cd49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-g8nq8\" (UID: \"27dc5a33-11f9-441d-9b56-0e6b33e3cd49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:38.867720 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:38.867637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:39.002506 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:39.002471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"]
Jul 16 22:55:39.004935 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:55:39.004907    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27dc5a33_11f9_441d_9b56_0e6b33e3cd49.slice/crio-4a5e84b8464b1f1166c01e74e53bfcf4a49a735cd2451b4ce4c60d90fe57c586 WatchSource:0}: Error finding container 4a5e84b8464b1f1166c01e74e53bfcf4a49a735cd2451b4ce4c60d90fe57c586: Status 404 returned error can't find the container with id 4a5e84b8464b1f1166c01e74e53bfcf4a49a735cd2451b4ce4c60d90fe57c586
Jul 16 22:55:39.596035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:39.596006    2564 scope.go:117] "RemoveContainer" containerID="c31e078627229f376df5e8f53af7a8b4d42ec63e6dee69fef7341d87252104fc"
Jul 16 22:55:39.850504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:39.850407    2564 generic.go:356] "Generic (PLEG): container finished" podID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerID="480589871e1949dcdc004d83e27fb7d3eb1fc14c485f4a8080909385a1c5b1be" exitCode=6
Jul 16 22:55:39.850504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:39.850452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" event={"ID":"27dc5a33-11f9-441d-9b56-0e6b33e3cd49","Type":"ContainerDied","Data":"480589871e1949dcdc004d83e27fb7d3eb1fc14c485f4a8080909385a1c5b1be"}
Jul 16 22:55:39.850749 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:39.850517    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" event={"ID":"27dc5a33-11f9-441d-9b56-0e6b33e3cd49","Type":"ContainerStarted","Data":"4a5e84b8464b1f1166c01e74e53bfcf4a49a735cd2451b4ce4c60d90fe57c586"}
Jul 16 22:55:39.850815 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:39.850753    2564 scope.go:117] "RemoveContainer" containerID="480589871e1949dcdc004d83e27fb7d3eb1fc14c485f4a8080909385a1c5b1be"
Jul 16 22:55:40.010185 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.010145    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"]
Jul 16 22:55:40.010519 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.010505    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:40.010564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.010523    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:40.010613 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.010603    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="968e9822-e1fa-4c2e-9a0c-7c558db809bb" containerName="revoke-keys"
Jul 16 22:55:40.013505 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.013489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:40.021952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.021926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"]
Jul 16 22:55:40.139202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.139076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb878507-5c48-40ac-bf43-1cc21a30c11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj\" (UID: \"eb878507-5c48-40ac-bf43-1cc21a30c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:40.240250 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.240206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb878507-5c48-40ac-bf43-1cc21a30c11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj\" (UID: \"eb878507-5c48-40ac-bf43-1cc21a30c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:40.240845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.240822    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb878507-5c48-40ac-bf43-1cc21a30c11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj\" (UID: \"eb878507-5c48-40ac-bf43-1cc21a30c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:40.324828 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.324800    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:40.465550 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.462207    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"]
Jul 16 22:55:40.858453 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.858420    2564 generic.go:356] "Generic (PLEG): container finished" podID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerID="c9768fd3753707cc88976d7e901eda756cce1382b3913d735f52eee5074ca9ae" exitCode=6
Jul 16 22:55:40.858914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.858488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" event={"ID":"1d6a31e8-4286-4fd6-86cd-e7af655220b7","Type":"ContainerDied","Data":"c9768fd3753707cc88976d7e901eda756cce1382b3913d735f52eee5074ca9ae"}
Jul 16 22:55:40.858914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.858540    2564 scope.go:117] "RemoveContainer" containerID="c31e078627229f376df5e8f53af7a8b4d42ec63e6dee69fef7341d87252104fc"
Jul 16 22:55:40.858914 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.858897    2564 scope.go:117] "RemoveContainer" containerID="c9768fd3753707cc88976d7e901eda756cce1382b3913d735f52eee5074ca9ae"
Jul 16 22:55:40.859184 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:40.859127    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7_odh-ai-gateway-infra(1d6a31e8-4286-4fd6-86cd-e7af655220b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7"
Jul 16 22:55:40.860068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.860040    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerStarted","Data":"1fc9f157c3d544bec8dd6c52714a03c45c4bfe32375f7540722ff8978e0b4c2f"}
Jul 16 22:55:40.860197 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.860090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerStarted","Data":"f4538b509d9fe212418dd40bf86f41558015dba55b60cc67e9ab06e5f330e0a5"}
Jul 16 22:55:40.861851 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.861824    2564 generic.go:356] "Generic (PLEG): container finished" podID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerID="613a6810f7330f538a48031662d2e8cd93834a5d1890f59f108845ea4fc1cead" exitCode=6
Jul 16 22:55:40.861952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.861861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" event={"ID":"27dc5a33-11f9-441d-9b56-0e6b33e3cd49","Type":"ContainerDied","Data":"613a6810f7330f538a48031662d2e8cd93834a5d1890f59f108845ea4fc1cead"}
Jul 16 22:55:40.862071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.862054    2564 scope.go:117] "RemoveContainer" containerID="613a6810f7330f538a48031662d2e8cd93834a5d1890f59f108845ea4fc1cead"
Jul 16 22:55:40.862262 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:40.862244    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-g8nq8_odh-ai-gateway-infra(27dc5a33-11f9-441d-9b56-0e6b33e3cd49)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49"
Jul 16 22:55:40.870222 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.870202    2564 scope.go:117] "RemoveContainer" containerID="480589871e1949dcdc004d83e27fb7d3eb1fc14c485f4a8080909385a1c5b1be"
Jul 16 22:55:40.890819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:40.890763    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" podStartSLOduration=1.890750159 podStartE2EDuration="1.890750159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:55:40.890455191 +0000 UTC m=+2297.211011263" watchObservedRunningTime="2026-07-16 22:55:40.890750159 +0000 UTC m=+2297.211306230"
Jul 16 22:55:41.869443 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:41.869403    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerID="1fc9f157c3d544bec8dd6c52714a03c45c4bfe32375f7540722ff8978e0b4c2f" exitCode=6
Jul 16 22:55:41.869875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:41.869487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerDied","Data":"1fc9f157c3d544bec8dd6c52714a03c45c4bfe32375f7540722ff8978e0b4c2f"}
Jul 16 22:55:41.869875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:41.869664    2564 scope.go:117] "RemoveContainer" containerID="1fc9f157c3d544bec8dd6c52714a03c45c4bfe32375f7540722ff8978e0b4c2f"
Jul 16 22:55:41.871196 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:41.871177    2564 scope.go:117] "RemoveContainer" containerID="613a6810f7330f538a48031662d2e8cd93834a5d1890f59f108845ea4fc1cead"
Jul 16 22:55:41.871360 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:41.871344    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-g8nq8_odh-ai-gateway-infra(27dc5a33-11f9-441d-9b56-0e6b33e3cd49)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49"
Jul 16 22:55:41.885744 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:41.885683    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"]
Jul 16 22:55:42.022858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.022833    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:42.158353 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.158263    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6a31e8-4286-4fd6-86cd-e7af655220b7-maas-api-service-ca\") pod \"1d6a31e8-4286-4fd6-86cd-e7af655220b7\" (UID: \"1d6a31e8-4286-4fd6-86cd-e7af655220b7\") "
Jul 16 22:55:42.158651 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.158628    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d6a31e8-4286-4fd6-86cd-e7af655220b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1d6a31e8-4286-4fd6-86cd-e7af655220b7" (UID: "1d6a31e8-4286-4fd6-86cd-e7af655220b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:42.259600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.259561    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6a31e8-4286-4fd6-86cd-e7af655220b7-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:55:42.879940 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.879864    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"
Jul 16 22:55:42.880353 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.879871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7" event={"ID":"1d6a31e8-4286-4fd6-86cd-e7af655220b7","Type":"ContainerDied","Data":"5104740ef907ead0c7de72340df45573cdb566cd850b2337bde3a523e8bb3a9d"}
Jul 16 22:55:42.880353 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.880016    2564 scope.go:117] "RemoveContainer" containerID="c9768fd3753707cc88976d7e901eda756cce1382b3913d735f52eee5074ca9ae"
Jul 16 22:55:42.881862 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.881836    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerID="05f18c985efd2b0e420a91beaea386b9c9470e92e92c7fc9c04f7579764093ec" exitCode=6
Jul 16 22:55:42.881966 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.881912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerDied","Data":"05f18c985efd2b0e420a91beaea386b9c9470e92e92c7fc9c04f7579764093ec"}
Jul 16 22:55:42.882219 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.882200    2564 scope.go:117] "RemoveContainer" containerID="05f18c985efd2b0e420a91beaea386b9c9470e92e92c7fc9c04f7579764093ec"
Jul 16 22:55:42.882519 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:42.882496    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj_odh-ai-gateway-infra(eb878507-5c48-40ac-bf43-1cc21a30c11e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e"
Jul 16 22:55:42.889712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.889692    2564 scope.go:117] "RemoveContainer" containerID="1fc9f157c3d544bec8dd6c52714a03c45c4bfe32375f7540722ff8978e0b4c2f"
Jul 16 22:55:42.912287 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.912259    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"]
Jul 16 22:55:42.916518 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:42.916493    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-qwcq7"]
Jul 16 22:55:43.601586 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:43.601555    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" path="/var/lib/kubelet/pods/1d6a31e8-4286-4fd6-86cd-e7af655220b7/volumes"
Jul 16 22:55:43.890306 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:43.890226    2564 scope.go:117] "RemoveContainer" containerID="05f18c985efd2b0e420a91beaea386b9c9470e92e92c7fc9c04f7579764093ec"
Jul 16 22:55:43.890676 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:43.890406    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj_odh-ai-gateway-infra(eb878507-5c48-40ac-bf43-1cc21a30c11e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e"
Jul 16 22:55:54.596493 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:54.596459    2564 scope.go:117] "RemoveContainer" containerID="05f18c985efd2b0e420a91beaea386b9c9470e92e92c7fc9c04f7579764093ec"
Jul 16 22:55:54.949882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:54.949845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerStarted","Data":"fded5298f24787b81e8d877fd686db6906e1276b10e5874a4e967f04f9256bcc"}
Jul 16 22:55:55.956465 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:55.956432    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerID="fded5298f24787b81e8d877fd686db6906e1276b10e5874a4e967f04f9256bcc" exitCode=6
Jul 16 22:55:55.956907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:55.956504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerDied","Data":"fded5298f24787b81e8d877fd686db6906e1276b10e5874a4e967f04f9256bcc"}
Jul 16 22:55:55.956907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:55.956565    2564 scope.go:117] "RemoveContainer" containerID="05f18c985efd2b0e420a91beaea386b9c9470e92e92c7fc9c04f7579764093ec"
Jul 16 22:55:55.956907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:55.956874    2564 scope.go:117] "RemoveContainer" containerID="fded5298f24787b81e8d877fd686db6906e1276b10e5874a4e967f04f9256bcc"
Jul 16 22:55:55.957091 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:55.957068    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj_odh-ai-gateway-infra(eb878507-5c48-40ac-bf43-1cc21a30c11e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e"
Jul 16 22:55:55.980570 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:55.980542    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"]
Jul 16 22:55:56.596819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.596784    2564 scope.go:117] "RemoveContainer" containerID="613a6810f7330f538a48031662d2e8cd93834a5d1890f59f108845ea4fc1cead"
Jul 16 22:55:56.713609 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713573    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"]
Jul 16 22:55:56.713935 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713923    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.713978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713936    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.713978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713957    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.713978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713963    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.713978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713974    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.714106 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.713981    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.714106 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.714025    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.714106 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.714033    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:55:56.718091 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.718073    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:55:56.726810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.726777    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"]
Jul 16 22:55:56.793362 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.793334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab629c-2f60-4b71-b320-ae95a2be81dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5\" (UID: \"1cab629c-2f60-4b71-b320-ae95a2be81dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:55:56.893886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.893852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab629c-2f60-4b71-b320-ae95a2be81dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5\" (UID: \"1cab629c-2f60-4b71-b320-ae95a2be81dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:55:56.894456 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.894430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab629c-2f60-4b71-b320-ae95a2be81dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5\" (UID: \"1cab629c-2f60-4b71-b320-ae95a2be81dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:55:56.964109 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.964074    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" event={"ID":"27dc5a33-11f9-441d-9b56-0e6b33e3cd49","Type":"ContainerStarted","Data":"ae0f5a7c0604bd6d5035ce5ef4013082a3b9e890f117b594fb235d95d0a342f1"}
Jul 16 22:55:56.988111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:56.988056    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" podStartSLOduration=18.988039928 podStartE2EDuration="18.988039928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:55:56.985622319 +0000 UTC m=+2313.306178391" watchObservedRunningTime="2026-07-16 22:55:56.988039928 +0000 UTC m=+2313.308596000"
Jul 16 22:55:57.030591 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.030331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:55:57.112116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.112086    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:57.170529 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.170498    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"]
Jul 16 22:55:57.171457 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:55:57.171432    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cab629c_2f60_4b71_b320_ae95a2be81dc.slice/crio-25a0845704d92910c138d3e3876599cf7887d46c740a90eaccca963d5dcd3262 WatchSource:0}: Error finding container 25a0845704d92910c138d3e3876599cf7887d46c740a90eaccca963d5dcd3262: Status 404 returned error can't find the container with id 25a0845704d92910c138d3e3876599cf7887d46c740a90eaccca963d5dcd3262
Jul 16 22:55:57.196992 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.196969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb878507-5c48-40ac-bf43-1cc21a30c11e-maas-api-service-ca\") pod \"eb878507-5c48-40ac-bf43-1cc21a30c11e\" (UID: \"eb878507-5c48-40ac-bf43-1cc21a30c11e\") "
Jul 16 22:55:57.197295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.197274    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb878507-5c48-40ac-bf43-1cc21a30c11e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb878507-5c48-40ac-bf43-1cc21a30c11e" (UID: "eb878507-5c48-40ac-bf43-1cc21a30c11e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:57.298731 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.298699    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb878507-5c48-40ac-bf43-1cc21a30c11e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:55:57.972295 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.972262    2564 generic.go:356] "Generic (PLEG): container finished" podID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerID="ae0f5a7c0604bd6d5035ce5ef4013082a3b9e890f117b594fb235d95d0a342f1" exitCode=6
Jul 16 22:55:57.972733 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.972339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" event={"ID":"27dc5a33-11f9-441d-9b56-0e6b33e3cd49","Type":"ContainerDied","Data":"ae0f5a7c0604bd6d5035ce5ef4013082a3b9e890f117b594fb235d95d0a342f1"}
Jul 16 22:55:57.972733 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.972415    2564 scope.go:117] "RemoveContainer" containerID="613a6810f7330f538a48031662d2e8cd93834a5d1890f59f108845ea4fc1cead"
Jul 16 22:55:57.972866 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.972751    2564 scope.go:117] "RemoveContainer" containerID="ae0f5a7c0604bd6d5035ce5ef4013082a3b9e890f117b594fb235d95d0a342f1"
Jul 16 22:55:57.972986 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:57.972962    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-g8nq8_odh-ai-gateway-infra(27dc5a33-11f9-441d-9b56-0e6b33e3cd49)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49"
Jul 16 22:55:57.973972 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.973945    2564 generic.go:356] "Generic (PLEG): container finished" podID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerID="4a4198e1f76d55ec64ac4d9e0d5b85fd3c6dfe779a6729f18a19427703cb09ca" exitCode=7
Jul 16 22:55:57.974084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.974029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" event={"ID":"1cab629c-2f60-4b71-b320-ae95a2be81dc","Type":"ContainerDied","Data":"4a4198e1f76d55ec64ac4d9e0d5b85fd3c6dfe779a6729f18a19427703cb09ca"}
Jul 16 22:55:57.974202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.974083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" event={"ID":"1cab629c-2f60-4b71-b320-ae95a2be81dc","Type":"ContainerStarted","Data":"25a0845704d92910c138d3e3876599cf7887d46c740a90eaccca963d5dcd3262"}
Jul 16 22:55:57.974202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.974159    2564 scope.go:117] "RemoveContainer" containerID="4a4198e1f76d55ec64ac4d9e0d5b85fd3c6dfe779a6729f18a19427703cb09ca"
Jul 16 22:55:57.975385 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.975364    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj" event={"ID":"eb878507-5c48-40ac-bf43-1cc21a30c11e","Type":"ContainerDied","Data":"f4538b509d9fe212418dd40bf86f41558015dba55b60cc67e9ab06e5f330e0a5"}
Jul 16 22:55:57.975385 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.975377    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"
Jul 16 22:55:57.985593 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.985576    2564 scope.go:117] "RemoveContainer" containerID="fded5298f24787b81e8d877fd686db6906e1276b10e5874a4e967f04f9256bcc"
Jul 16 22:55:57.997716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:57.997689    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"]
Jul 16 22:55:58.007808 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:58.007787    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"]
Jul 16 22:55:58.013333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:58.013263    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-wt8cj"]
Jul 16 22:55:58.986032 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:58.986004    2564 generic.go:356] "Generic (PLEG): container finished" podID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerID="03748cfd74f170ba17742cfa9518f477bda59433fbb7ab6235fadf699b5c0f2e" exitCode=7
Jul 16 22:55:58.986414 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:58.986082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" event={"ID":"1cab629c-2f60-4b71-b320-ae95a2be81dc","Type":"ContainerDied","Data":"03748cfd74f170ba17742cfa9518f477bda59433fbb7ab6235fadf699b5c0f2e"}
Jul 16 22:55:58.986414 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:58.986157    2564 scope.go:117] "RemoveContainer" containerID="4a4198e1f76d55ec64ac4d9e0d5b85fd3c6dfe779a6729f18a19427703cb09ca"
Jul 16 22:55:58.986414 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:58.986385    2564 scope.go:117] "RemoveContainer" containerID="03748cfd74f170ba17742cfa9518f477bda59433fbb7ab6235fadf699b5c0f2e"
Jul 16 22:55:58.986767 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:55:58.986731    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5_odh-ai-gateway-infra(1cab629c-2f60-4b71-b320-ae95a2be81dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc"
Jul 16 22:55:59.114090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.114065    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:59.214684 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.214650    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27dc5a33-11f9-441d-9b56-0e6b33e3cd49-maas-api-service-ca\") pod \"27dc5a33-11f9-441d-9b56-0e6b33e3cd49\" (UID: \"27dc5a33-11f9-441d-9b56-0e6b33e3cd49\") "
Jul 16 22:55:59.215023 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.215001    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27dc5a33-11f9-441d-9b56-0e6b33e3cd49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27dc5a33-11f9-441d-9b56-0e6b33e3cd49" (UID: "27dc5a33-11f9-441d-9b56-0e6b33e3cd49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:59.316195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.316093    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27dc5a33-11f9-441d-9b56-0e6b33e3cd49-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:55:59.602008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.601930    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" path="/var/lib/kubelet/pods/eb878507-5c48-40ac-bf43-1cc21a30c11e/volumes"
Jul 16 22:55:59.998507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.998475    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"
Jul 16 22:55:59.998937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.998469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8" event={"ID":"27dc5a33-11f9-441d-9b56-0e6b33e3cd49","Type":"ContainerDied","Data":"4a5e84b8464b1f1166c01e74e53bfcf4a49a735cd2451b4ce4c60d90fe57c586"}
Jul 16 22:55:59.998937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:55:59.998654    2564 scope.go:117] "RemoveContainer" containerID="ae0f5a7c0604bd6d5035ce5ef4013082a3b9e890f117b594fb235d95d0a342f1"
Jul 16 22:56:00.000560 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:00.000538    2564 scope.go:117] "RemoveContainer" containerID="03748cfd74f170ba17742cfa9518f477bda59433fbb7ab6235fadf699b5c0f2e"
Jul 16 22:56:00.000833 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:00.000806    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5_odh-ai-gateway-infra(1cab629c-2f60-4b71-b320-ae95a2be81dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc"
Jul 16 22:56:00.031274 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:00.031241    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"]
Jul 16 22:56:00.037228 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:00.037203    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-g8nq8"]
Jul 16 22:56:01.602081 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:01.602048    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" path="/var/lib/kubelet/pods/27dc5a33-11f9-441d-9b56-0e6b33e3cd49/volumes"
Jul 16 22:56:12.596690 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:12.596653    2564 scope.go:117] "RemoveContainer" containerID="03748cfd74f170ba17742cfa9518f477bda59433fbb7ab6235fadf699b5c0f2e"
Jul 16 22:56:13.072743 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:13.072708    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" event={"ID":"1cab629c-2f60-4b71-b320-ae95a2be81dc","Type":"ContainerStarted","Data":"fd7246d9b47934cb6d175550c303486462776f5fb9d058f5c0ef2a64b0437ba4"}
Jul 16 22:56:13.089712 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:13.089628    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" podStartSLOduration=17.089611701 podStartE2EDuration="17.089611701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:55:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:56:13.088289564 +0000 UTC m=+2329.408845630" watchObservedRunningTime="2026-07-16 22:56:13.089611701 +0000 UTC m=+2329.410167772"
Jul 16 22:56:14.080201 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:14.080166    2564 generic.go:356] "Generic (PLEG): container finished" podID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerID="fd7246d9b47934cb6d175550c303486462776f5fb9d058f5c0ef2a64b0437ba4" exitCode=7
Jul 16 22:56:14.080676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:14.080247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" event={"ID":"1cab629c-2f60-4b71-b320-ae95a2be81dc","Type":"ContainerDied","Data":"fd7246d9b47934cb6d175550c303486462776f5fb9d058f5c0ef2a64b0437ba4"}
Jul 16 22:56:14.080676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:14.080321    2564 scope.go:117] "RemoveContainer" containerID="03748cfd74f170ba17742cfa9518f477bda59433fbb7ab6235fadf699b5c0f2e"
Jul 16 22:56:14.080676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:14.080626    2564 scope.go:117] "RemoveContainer" containerID="fd7246d9b47934cb6d175550c303486462776f5fb9d058f5c0ef2a64b0437ba4"
Jul 16 22:56:14.080887 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:14.080859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5_odh-ai-gateway-infra(1cab629c-2f60-4b71-b320-ae95a2be81dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc"
Jul 16 22:56:14.118614 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:14.118576    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"]
Jul 16 22:56:15.219663 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:15.219635    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:56:15.356869 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:15.356782    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab629c-2f60-4b71-b320-ae95a2be81dc-maas-api-service-ca\") pod \"1cab629c-2f60-4b71-b320-ae95a2be81dc\" (UID: \"1cab629c-2f60-4b71-b320-ae95a2be81dc\") "
Jul 16 22:56:15.357104 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:15.357081    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cab629c-2f60-4b71-b320-ae95a2be81dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1cab629c-2f60-4b71-b320-ae95a2be81dc" (UID: "1cab629c-2f60-4b71-b320-ae95a2be81dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:56:15.458578 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:15.458544    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cab629c-2f60-4b71-b320-ae95a2be81dc-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:56:16.093945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:16.093915    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"
Jul 16 22:56:16.093945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:16.093928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5" event={"ID":"1cab629c-2f60-4b71-b320-ae95a2be81dc","Type":"ContainerDied","Data":"25a0845704d92910c138d3e3876599cf7887d46c740a90eaccca963d5dcd3262"}
Jul 16 22:56:16.094193 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:16.093990    2564 scope.go:117] "RemoveContainer" containerID="fd7246d9b47934cb6d175550c303486462776f5fb9d058f5c0ef2a64b0437ba4"
Jul 16 22:56:16.115718 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:16.115683    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"]
Jul 16 22:56:16.119638 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:16.119610    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-69cb5"]
Jul 16 22:56:17.601541 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:17.601506    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" path="/var/lib/kubelet/pods/1cab629c-2f60-4b71-b320-ae95a2be81dc/volumes"
Jul 16 22:56:22.065108 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065068    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"]
Jul 16 22:56:22.065622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065582    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:22.065622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065603    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:22.065622 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065618    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065627    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065645    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065654    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065663    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065672    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065686    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065694    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065723    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.065796 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065732    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065813    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065826    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065837    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065847    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065854    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065863    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065874    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065883    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:22.066251 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.065892    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6a31e8-4286-4fd6-86cd-e7af655220b7" containerName="revoke-keys"
Jul 16 22:56:22.070580 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.070555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:22.073111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.073086    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:56:22.073276 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.073240    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:56:22.077613 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.077585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"]
Jul 16 22:56:22.218241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.218200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e0eda1-9748-4944-9af8-8479974e31c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj\" (UID: \"66e0eda1-9748-4944-9af8-8479974e31c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:22.318973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.318881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e0eda1-9748-4944-9af8-8479974e31c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj\" (UID: \"66e0eda1-9748-4944-9af8-8479974e31c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:22.319545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.319523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e0eda1-9748-4944-9af8-8479974e31c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj\" (UID: \"66e0eda1-9748-4944-9af8-8479974e31c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:22.383789 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.383754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:22.520126 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:22.520088    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"]
Jul 16 22:56:22.523126 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:56:22.523095    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e0eda1_9748_4944_9af8_8479974e31c1.slice/crio-288fd7fa7e122e49eb0f13b86d8845fd1c72370736a9629827d174bd47eaf30b WatchSource:0}: Error finding container 288fd7fa7e122e49eb0f13b86d8845fd1c72370736a9629827d174bd47eaf30b: Status 404 returned error can't find the container with id 288fd7fa7e122e49eb0f13b86d8845fd1c72370736a9629827d174bd47eaf30b
Jul 16 22:56:23.132936 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:23.132903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerStarted","Data":"783aaf9e159b65613c86819f0cad04ff6adecc246665bcd7abeaf228b7bf0975"}
Jul 16 22:56:23.133369 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:23.132958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerStarted","Data":"288fd7fa7e122e49eb0f13b86d8845fd1c72370736a9629827d174bd47eaf30b"}
Jul 16 22:56:23.151270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:23.151222    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" podStartSLOduration=1.15120255 podStartE2EDuration="1.15120255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:56:23.150814959 +0000 UTC m=+2339.471371032" watchObservedRunningTime="2026-07-16 22:56:23.15120255 +0000 UTC m=+2339.471758622"
Jul 16 22:56:24.139208 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:24.139171    2564 generic.go:356] "Generic (PLEG): container finished" podID="66e0eda1-9748-4944-9af8-8479974e31c1" containerID="783aaf9e159b65613c86819f0cad04ff6adecc246665bcd7abeaf228b7bf0975" exitCode=6
Jul 16 22:56:24.139688 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:24.139263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerDied","Data":"783aaf9e159b65613c86819f0cad04ff6adecc246665bcd7abeaf228b7bf0975"}
Jul 16 22:56:24.139688 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:24.139443    2564 scope.go:117] "RemoveContainer" containerID="783aaf9e159b65613c86819f0cad04ff6adecc246665bcd7abeaf228b7bf0975"
Jul 16 22:56:25.146324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:25.146226    2564 generic.go:356] "Generic (PLEG): container finished" podID="66e0eda1-9748-4944-9af8-8479974e31c1" containerID="5bbb2f0b04676ed19f086d874e8f0d40e2905a26463ad7d633ee59e108131f10" exitCode=6
Jul 16 22:56:25.146324 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:25.146309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerDied","Data":"5bbb2f0b04676ed19f086d874e8f0d40e2905a26463ad7d633ee59e108131f10"}
Jul 16 22:56:25.146831 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:25.146381    2564 scope.go:117] "RemoveContainer" containerID="783aaf9e159b65613c86819f0cad04ff6adecc246665bcd7abeaf228b7bf0975"
Jul 16 22:56:25.146831 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:25.146515    2564 scope.go:117] "RemoveContainer" containerID="5bbb2f0b04676ed19f086d874e8f0d40e2905a26463ad7d633ee59e108131f10"
Jul 16 22:56:25.146831 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:25.146744    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj_odh-ai-gateway-infra(66e0eda1-9748-4944-9af8-8479974e31c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" podUID="66e0eda1-9748-4944-9af8-8479974e31c1"
Jul 16 22:56:26.152934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:26.152902    2564 scope.go:117] "RemoveContainer" containerID="5bbb2f0b04676ed19f086d874e8f0d40e2905a26463ad7d633ee59e108131f10"
Jul 16 22:56:26.153355 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:26.153124    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj_odh-ai-gateway-infra(66e0eda1-9748-4944-9af8-8479974e31c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" podUID="66e0eda1-9748-4944-9af8-8479974e31c1"
Jul 16 22:56:30.055195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055154    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"]
Jul 16 22:56:30.055574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055528    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:30.055574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055539    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="27dc5a33-11f9-441d-9b56-0e6b33e3cd49" containerName="revoke-keys"
Jul 16 22:56:30.055574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055551    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:30.055574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055557    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:30.055574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055566    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:30.055574 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055571    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cab629c-2f60-4b71-b320-ae95a2be81dc" containerName="revoke-keys"
Jul 16 22:56:30.055824 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.055632    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb878507-5c48-40ac-bf43-1cc21a30c11e" containerName="revoke-keys"
Jul 16 22:56:30.058587 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.058570    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:30.066848 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.066816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"]
Jul 16 22:56:30.190908 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.190870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/847a482f-f435-48e2-b6c6-07de8a3ef219-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k\" (UID: \"847a482f-f435-48e2-b6c6-07de8a3ef219\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:30.292152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.292097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/847a482f-f435-48e2-b6c6-07de8a3ef219-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k\" (UID: \"847a482f-f435-48e2-b6c6-07de8a3ef219\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:30.292724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.292705    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/847a482f-f435-48e2-b6c6-07de8a3ef219-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k\" (UID: \"847a482f-f435-48e2-b6c6-07de8a3ef219\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:30.370289 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.370193    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:30.510627 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:30.510597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"]
Jul 16 22:56:30.512936 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:56:30.512908    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847a482f_f435_48e2_b6c6_07de8a3ef219.slice/crio-e870b2ea297e187ab4e72fbaad36f5a96db10da6b5998c2f05be201c70d02700 WatchSource:0}: Error finding container e870b2ea297e187ab4e72fbaad36f5a96db10da6b5998c2f05be201c70d02700: Status 404 returned error can't find the container with id e870b2ea297e187ab4e72fbaad36f5a96db10da6b5998c2f05be201c70d02700
Jul 16 22:56:31.179466 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:31.179429    2564 generic.go:356] "Generic (PLEG): container finished" podID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerID="62aa3526b5d22ad844956b99d88828055e86c925ca1abbfa6f3705e3ca1e7379" exitCode=6
Jul 16 22:56:31.179919 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:31.179470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" event={"ID":"847a482f-f435-48e2-b6c6-07de8a3ef219","Type":"ContainerDied","Data":"62aa3526b5d22ad844956b99d88828055e86c925ca1abbfa6f3705e3ca1e7379"}
Jul 16 22:56:31.179919 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:31.179532    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" event={"ID":"847a482f-f435-48e2-b6c6-07de8a3ef219","Type":"ContainerStarted","Data":"e870b2ea297e187ab4e72fbaad36f5a96db10da6b5998c2f05be201c70d02700"}
Jul 16 22:56:31.179919 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:31.179811    2564 scope.go:117] "RemoveContainer" containerID="62aa3526b5d22ad844956b99d88828055e86c925ca1abbfa6f3705e3ca1e7379"
Jul 16 22:56:32.187076 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:32.186977    2564 generic.go:356] "Generic (PLEG): container finished" podID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerID="7a5f65a7e6915ee6ac4f7180678f25c9425d0fe5cbdf11ac5fda584df4d43d63" exitCode=6
Jul 16 22:56:32.187545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:32.187068    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" event={"ID":"847a482f-f435-48e2-b6c6-07de8a3ef219","Type":"ContainerDied","Data":"7a5f65a7e6915ee6ac4f7180678f25c9425d0fe5cbdf11ac5fda584df4d43d63"}
Jul 16 22:56:32.187545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:32.187125    2564 scope.go:117] "RemoveContainer" containerID="62aa3526b5d22ad844956b99d88828055e86c925ca1abbfa6f3705e3ca1e7379"
Jul 16 22:56:32.187545 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:32.187240    2564 scope.go:117] "RemoveContainer" containerID="7a5f65a7e6915ee6ac4f7180678f25c9425d0fe5cbdf11ac5fda584df4d43d63"
Jul 16 22:56:32.187545 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:32.187474    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k_odh-ai-gateway-infra(847a482f-f435-48e2-b6c6-07de8a3ef219)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219"
Jul 16 22:56:33.193961 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:33.193932    2564 scope.go:117] "RemoveContainer" containerID="7a5f65a7e6915ee6ac4f7180678f25c9425d0fe5cbdf11ac5fda584df4d43d63"
Jul 16 22:56:33.194389 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:33.194126    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k_odh-ai-gateway-infra(847a482f-f435-48e2-b6c6-07de8a3ef219)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219"
Jul 16 22:56:38.610149 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.610096    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"]
Jul 16 22:56:38.614705 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.614686    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:38.621383 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.621356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"]
Jul 16 22:56:38.773853 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.773806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961c8d8b-2570-47f0-8308-d69a07226f55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-kl99v\" (UID: \"961c8d8b-2570-47f0-8308-d69a07226f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:38.874657 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.874557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961c8d8b-2570-47f0-8308-d69a07226f55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-kl99v\" (UID: \"961c8d8b-2570-47f0-8308-d69a07226f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:38.875199 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.875180    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961c8d8b-2570-47f0-8308-d69a07226f55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-kl99v\" (UID: \"961c8d8b-2570-47f0-8308-d69a07226f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:38.926110 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:38.926063    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:39.085105 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:39.085081    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"]
Jul 16 22:56:39.087110 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:56:39.087082    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961c8d8b_2570_47f0_8308_d69a07226f55.slice/crio-2b91cd1088418c4e9b2c8cc564984db2f0734ea5718c4840c76bf54263c804ef WatchSource:0}: Error finding container 2b91cd1088418c4e9b2c8cc564984db2f0734ea5718c4840c76bf54263c804ef: Status 404 returned error can't find the container with id 2b91cd1088418c4e9b2c8cc564984db2f0734ea5718c4840c76bf54263c804ef
Jul 16 22:56:39.227124 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:39.227090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" event={"ID":"961c8d8b-2570-47f0-8308-d69a07226f55","Type":"ContainerStarted","Data":"2b91cd1088418c4e9b2c8cc564984db2f0734ea5718c4840c76bf54263c804ef"}
Jul 16 22:56:39.596283 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:39.596246    2564 scope.go:117] "RemoveContainer" containerID="5bbb2f0b04676ed19f086d874e8f0d40e2905a26463ad7d633ee59e108131f10"
Jul 16 22:56:40.235158 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:40.235101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerStarted","Data":"4f2371d72e15eb2ffd6431b7d37253d155eade7013743cbd890f8119b4019ca0"}
Jul 16 22:56:40.236607 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:40.236581    2564 generic.go:356] "Generic (PLEG): container finished" podID="961c8d8b-2570-47f0-8308-d69a07226f55" containerID="1ec3f04f3efeda47eb6fbce3a9be7f28c44c29ba1a89eb77eb6f1a933eff2f1d" exitCode=6
Jul 16 22:56:40.236726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:40.236668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" event={"ID":"961c8d8b-2570-47f0-8308-d69a07226f55","Type":"ContainerDied","Data":"1ec3f04f3efeda47eb6fbce3a9be7f28c44c29ba1a89eb77eb6f1a933eff2f1d"}
Jul 16 22:56:40.236811 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:40.236797    2564 scope.go:117] "RemoveContainer" containerID="1ec3f04f3efeda47eb6fbce3a9be7f28c44c29ba1a89eb77eb6f1a933eff2f1d"
Jul 16 22:56:41.250345 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.250304    2564 generic.go:356] "Generic (PLEG): container finished" podID="66e0eda1-9748-4944-9af8-8479974e31c1" containerID="4f2371d72e15eb2ffd6431b7d37253d155eade7013743cbd890f8119b4019ca0" exitCode=6
Jul 16 22:56:41.250845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.250338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerDied","Data":"4f2371d72e15eb2ffd6431b7d37253d155eade7013743cbd890f8119b4019ca0"}
Jul 16 22:56:41.250845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.250439    2564 scope.go:117] "RemoveContainer" containerID="5bbb2f0b04676ed19f086d874e8f0d40e2905a26463ad7d633ee59e108131f10"
Jul 16 22:56:41.250845 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.250736    2564 scope.go:117] "RemoveContainer" containerID="4f2371d72e15eb2ffd6431b7d37253d155eade7013743cbd890f8119b4019ca0"
Jul 16 22:56:41.251006 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:41.250984    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj_odh-ai-gateway-infra(66e0eda1-9748-4944-9af8-8479974e31c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" podUID="66e0eda1-9748-4944-9af8-8479974e31c1"
Jul 16 22:56:41.252423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.252403    2564 generic.go:356] "Generic (PLEG): container finished" podID="961c8d8b-2570-47f0-8308-d69a07226f55" containerID="d7ebd5b76c46ed299e1daa729b1c9d489574b2d4cd8a4d42bf46a0a3b071db80" exitCode=6
Jul 16 22:56:41.252523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.252473    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" event={"ID":"961c8d8b-2570-47f0-8308-d69a07226f55","Type":"ContainerDied","Data":"d7ebd5b76c46ed299e1daa729b1c9d489574b2d4cd8a4d42bf46a0a3b071db80"}
Jul 16 22:56:41.252643 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.252631    2564 scope.go:117] "RemoveContainer" containerID="d7ebd5b76c46ed299e1daa729b1c9d489574b2d4cd8a4d42bf46a0a3b071db80"
Jul 16 22:56:41.252792 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:41.252778    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-kl99v_odh-ai-gateway-infra(961c8d8b-2570-47f0-8308-d69a07226f55)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" podUID="961c8d8b-2570-47f0-8308-d69a07226f55"
Jul 16 22:56:41.262357 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.262338    2564 scope.go:117] "RemoveContainer" containerID="1ec3f04f3efeda47eb6fbce3a9be7f28c44c29ba1a89eb77eb6f1a933eff2f1d"
Jul 16 22:56:41.264318 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:41.264296    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"]
Jul 16 22:56:42.261337 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:42.261317    2564 scope.go:117] "RemoveContainer" containerID="d7ebd5b76c46ed299e1daa729b1c9d489574b2d4cd8a4d42bf46a0a3b071db80"
Jul 16 22:56:42.261643 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:42.261522    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-kl99v_odh-ai-gateway-infra(961c8d8b-2570-47f0-8308-d69a07226f55)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" podUID="961c8d8b-2570-47f0-8308-d69a07226f55"
Jul 16 22:56:42.398986 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:42.398957    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:42.406052 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:42.406029    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e0eda1-9748-4944-9af8-8479974e31c1-maas-api-service-ca\") pod \"66e0eda1-9748-4944-9af8-8479974e31c1\" (UID: \"66e0eda1-9748-4944-9af8-8479974e31c1\") "
Jul 16 22:56:42.406386 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:42.406363    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66e0eda1-9748-4944-9af8-8479974e31c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66e0eda1-9748-4944-9af8-8479974e31c1" (UID: "66e0eda1-9748-4944-9af8-8479974e31c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:56:42.507189 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:42.507125    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66e0eda1-9748-4944-9af8-8479974e31c1-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:56:43.267656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:43.267625    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"
Jul 16 22:56:43.268165 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:43.267618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj" event={"ID":"66e0eda1-9748-4944-9af8-8479974e31c1","Type":"ContainerDied","Data":"288fd7fa7e122e49eb0f13b86d8845fd1c72370736a9629827d174bd47eaf30b"}
Jul 16 22:56:43.268165 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:43.267776    2564 scope.go:117] "RemoveContainer" containerID="4f2371d72e15eb2ffd6431b7d37253d155eade7013743cbd890f8119b4019ca0"
Jul 16 22:56:43.302580 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:43.302554    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"]
Jul 16 22:56:43.314802 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:43.314773    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-9nqhj"]
Jul 16 22:56:43.601907 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:43.601831    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" path="/var/lib/kubelet/pods/66e0eda1-9748-4944-9af8-8479974e31c1/volumes"
Jul 16 22:56:44.596564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:44.596524    2564 scope.go:117] "RemoveContainer" containerID="7a5f65a7e6915ee6ac4f7180678f25c9425d0fe5cbdf11ac5fda584df4d43d63"
Jul 16 22:56:45.280598 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:45.280569    2564 generic.go:356] "Generic (PLEG): container finished" podID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerID="6376ece57596d6599fa138cf3da96b347fd0a4bd585a80a15c4d6463d0e46a77" exitCode=6
Jul 16 22:56:45.280774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:45.280644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" event={"ID":"847a482f-f435-48e2-b6c6-07de8a3ef219","Type":"ContainerDied","Data":"6376ece57596d6599fa138cf3da96b347fd0a4bd585a80a15c4d6463d0e46a77"}
Jul 16 22:56:45.280774 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:45.280706    2564 scope.go:117] "RemoveContainer" containerID="7a5f65a7e6915ee6ac4f7180678f25c9425d0fe5cbdf11ac5fda584df4d43d63"
Jul 16 22:56:45.281021 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:45.280997    2564 scope.go:117] "RemoveContainer" containerID="6376ece57596d6599fa138cf3da96b347fd0a4bd585a80a15c4d6463d0e46a77"
Jul 16 22:56:45.281288 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:45.281268    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k_odh-ai-gateway-infra(847a482f-f435-48e2-b6c6-07de8a3ef219)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219"
Jul 16 22:56:46.309203 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.309168    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"]
Jul 16 22:56:46.448770 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.448743    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:46.544246 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.544197    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/847a482f-f435-48e2-b6c6-07de8a3ef219-maas-api-service-ca\") pod \"847a482f-f435-48e2-b6c6-07de8a3ef219\" (UID: \"847a482f-f435-48e2-b6c6-07de8a3ef219\") "
Jul 16 22:56:46.544605 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.544577    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/847a482f-f435-48e2-b6c6-07de8a3ef219-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "847a482f-f435-48e2-b6c6-07de8a3ef219" (UID: "847a482f-f435-48e2-b6c6-07de8a3ef219"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:56:46.645637 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.645546    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/847a482f-f435-48e2-b6c6-07de8a3ef219-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:56:46.764633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764599    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"]
Jul 16 22:56:46.764945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764933    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:56:46.764989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764947    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:56:46.764989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764960    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:56:46.764989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764965    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:56:46.764989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764978    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.764989 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764983    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.764994    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765001    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765010    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765015    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765087    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765096    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765101    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765108    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66e0eda1-9748-4944-9af8-8479974e31c1" containerName="revoke-keys"
Jul 16 22:56:46.765173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.765113    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:56:46.768227 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.768210    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:56:46.776995 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.776964    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"]
Jul 16 22:56:46.847126 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.847089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae968fd-2497-4aaa-b779-25a03c9bb481-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb\" (UID: \"0ae968fd-2497-4aaa-b779-25a03c9bb481\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:56:46.947655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.947623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae968fd-2497-4aaa-b779-25a03c9bb481-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb\" (UID: \"0ae968fd-2497-4aaa-b779-25a03c9bb481\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:56:46.948323 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:46.948303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae968fd-2497-4aaa-b779-25a03c9bb481-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb\" (UID: \"0ae968fd-2497-4aaa-b779-25a03c9bb481\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:56:47.079556 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.079519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:56:47.227489 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.227459    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"]
Jul 16 22:56:47.229111 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:56:47.229084    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae968fd_2497_4aaa_b779_25a03c9bb481.slice/crio-e8f1f16c08ae3b9d460d7f044e732d0ad251cb2c4468a6ee45119a4c707a5076 WatchSource:0}: Error finding container e8f1f16c08ae3b9d460d7f044e732d0ad251cb2c4468a6ee45119a4c707a5076: Status 404 returned error can't find the container with id e8f1f16c08ae3b9d460d7f044e732d0ad251cb2c4468a6ee45119a4c707a5076
Jul 16 22:56:47.299450 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.299423    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"
Jul 16 22:56:47.299450 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.299445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k" event={"ID":"847a482f-f435-48e2-b6c6-07de8a3ef219","Type":"ContainerDied","Data":"e870b2ea297e187ab4e72fbaad36f5a96db10da6b5998c2f05be201c70d02700"}
Jul 16 22:56:47.299702 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.299507    2564 scope.go:117] "RemoveContainer" containerID="6376ece57596d6599fa138cf3da96b347fd0a4bd585a80a15c4d6463d0e46a77"
Jul 16 22:56:47.300766 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.300736    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" event={"ID":"0ae968fd-2497-4aaa-b779-25a03c9bb481","Type":"ContainerStarted","Data":"e8f1f16c08ae3b9d460d7f044e732d0ad251cb2c4468a6ee45119a4c707a5076"}
Jul 16 22:56:47.322724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.322660    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"]
Jul 16 22:56:47.328476 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.328454    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-zlk2k"]
Jul 16 22:56:47.601487 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:47.601455    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" path="/var/lib/kubelet/pods/847a482f-f435-48e2-b6c6-07de8a3ef219/volumes"
Jul 16 22:56:48.308481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:48.308447    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerID="34008ed96952897c7217b78109d6824e59918b6a09e9a4d8e04d43156bbf62ec" exitCode=7
Jul 16 22:56:48.308676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:48.308537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" event={"ID":"0ae968fd-2497-4aaa-b779-25a03c9bb481","Type":"ContainerDied","Data":"34008ed96952897c7217b78109d6824e59918b6a09e9a4d8e04d43156bbf62ec"}
Jul 16 22:56:48.308730 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:48.308706    2564 scope.go:117] "RemoveContainer" containerID="34008ed96952897c7217b78109d6824e59918b6a09e9a4d8e04d43156bbf62ec"
Jul 16 22:56:49.315401 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:49.315362    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerID="e5e73230e1060024c2730187caac579355ca4ca93d984e407cee8d3e096b3857" exitCode=7
Jul 16 22:56:49.315821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:49.315435    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" event={"ID":"0ae968fd-2497-4aaa-b779-25a03c9bb481","Type":"ContainerDied","Data":"e5e73230e1060024c2730187caac579355ca4ca93d984e407cee8d3e096b3857"}
Jul 16 22:56:49.315821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:49.315500    2564 scope.go:117] "RemoveContainer" containerID="34008ed96952897c7217b78109d6824e59918b6a09e9a4d8e04d43156bbf62ec"
Jul 16 22:56:49.315821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:49.315684    2564 scope.go:117] "RemoveContainer" containerID="e5e73230e1060024c2730187caac579355ca4ca93d984e407cee8d3e096b3857"
Jul 16 22:56:49.315973 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:49.315898    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb_odh-ai-gateway-infra(0ae968fd-2497-4aaa-b779-25a03c9bb481)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481"
Jul 16 22:56:50.322790 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:50.322758    2564 scope.go:117] "RemoveContainer" containerID="e5e73230e1060024c2730187caac579355ca4ca93d984e407cee8d3e096b3857"
Jul 16 22:56:50.323215 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:50.322959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb_odh-ai-gateway-infra(0ae968fd-2497-4aaa-b779-25a03c9bb481)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481"
Jul 16 22:56:55.599478 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:55.599446    2564 scope.go:117] "RemoveContainer" containerID="d7ebd5b76c46ed299e1daa729b1c9d489574b2d4cd8a4d42bf46a0a3b071db80"
Jul 16 22:56:56.356107 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:56.356071    2564 generic.go:356] "Generic (PLEG): container finished" podID="961c8d8b-2570-47f0-8308-d69a07226f55" containerID="5f412088372f0312022aab0b4fe7a8404e50586201df3269dabc20b924f9db65" exitCode=6
Jul 16 22:56:56.356298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:56.356117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" event={"ID":"961c8d8b-2570-47f0-8308-d69a07226f55","Type":"ContainerDied","Data":"5f412088372f0312022aab0b4fe7a8404e50586201df3269dabc20b924f9db65"}
Jul 16 22:56:56.356298 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:56.356178    2564 scope.go:117] "RemoveContainer" containerID="d7ebd5b76c46ed299e1daa729b1c9d489574b2d4cd8a4d42bf46a0a3b071db80"
Jul 16 22:56:56.356467 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:56.356451    2564 scope.go:117] "RemoveContainer" containerID="5f412088372f0312022aab0b4fe7a8404e50586201df3269dabc20b924f9db65"
Jul 16 22:56:56.356661 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:56:56.356643    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-kl99v_odh-ai-gateway-infra(961c8d8b-2570-47f0-8308-d69a07226f55)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" podUID="961c8d8b-2570-47f0-8308-d69a07226f55"
Jul 16 22:56:57.388878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:57.384687    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"]
Jul 16 22:56:57.526277 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:57.526246    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:57.637147 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:57.637110    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961c8d8b-2570-47f0-8308-d69a07226f55-maas-api-service-ca\") pod \"961c8d8b-2570-47f0-8308-d69a07226f55\" (UID: \"961c8d8b-2570-47f0-8308-d69a07226f55\") "
Jul 16 22:56:57.637475 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:57.637449    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/961c8d8b-2570-47f0-8308-d69a07226f55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "961c8d8b-2570-47f0-8308-d69a07226f55" (UID: "961c8d8b-2570-47f0-8308-d69a07226f55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:56:57.738657 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:57.738613    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/961c8d8b-2570-47f0-8308-d69a07226f55-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:56:58.369711 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:58.369677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v" event={"ID":"961c8d8b-2570-47f0-8308-d69a07226f55","Type":"ContainerDied","Data":"2b91cd1088418c4e9b2c8cc564984db2f0734ea5718c4840c76bf54263c804ef"}
Jul 16 22:56:58.369711 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:58.369710    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"
Jul 16 22:56:58.369918 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:58.369740    2564 scope.go:117] "RemoveContainer" containerID="5f412088372f0312022aab0b4fe7a8404e50586201df3269dabc20b924f9db65"
Jul 16 22:56:58.393610 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:58.393584    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"]
Jul 16 22:56:58.399149 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:58.399105    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-kl99v"]
Jul 16 22:56:59.601827 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:56:59.601793    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" path="/var/lib/kubelet/pods/961c8d8b-2570-47f0-8308-d69a07226f55/volumes"
Jul 16 22:57:03.595956 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:03.595929    2564 scope.go:117] "RemoveContainer" containerID="e5e73230e1060024c2730187caac579355ca4ca93d984e407cee8d3e096b3857"
Jul 16 22:57:04.411752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:04.411720    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerID="2414f2ac4619f8248c15570d1d989b5a259b50c39036c11b052214f600e20268" exitCode=7
Jul 16 22:57:04.411898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:04.411790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" event={"ID":"0ae968fd-2497-4aaa-b779-25a03c9bb481","Type":"ContainerDied","Data":"2414f2ac4619f8248c15570d1d989b5a259b50c39036c11b052214f600e20268"}
Jul 16 22:57:04.411898 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:04.411848    2564 scope.go:117] "RemoveContainer" containerID="e5e73230e1060024c2730187caac579355ca4ca93d984e407cee8d3e096b3857"
Jul 16 22:57:04.412167 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:04.412126    2564 scope.go:117] "RemoveContainer" containerID="2414f2ac4619f8248c15570d1d989b5a259b50c39036c11b052214f600e20268"
Jul 16 22:57:04.412359 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:04.412339    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb_odh-ai-gateway-infra(0ae968fd-2497-4aaa-b779-25a03c9bb481)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481"
Jul 16 22:57:05.439083 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:05.439043    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"]
Jul 16 22:57:05.570215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:05.570192    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:57:05.712937 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:05.712901    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae968fd-2497-4aaa-b779-25a03c9bb481-maas-api-service-ca\") pod \"0ae968fd-2497-4aaa-b779-25a03c9bb481\" (UID: \"0ae968fd-2497-4aaa-b779-25a03c9bb481\") "
Jul 16 22:57:05.713279 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:05.713257    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae968fd-2497-4aaa-b779-25a03c9bb481-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ae968fd-2497-4aaa-b779-25a03c9bb481" (UID: "0ae968fd-2497-4aaa-b779-25a03c9bb481"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:05.814420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:05.814379    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae968fd-2497-4aaa-b779-25a03c9bb481-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:06.424365 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.424335    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"
Jul 16 22:57:06.424365 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.424354    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb" event={"ID":"0ae968fd-2497-4aaa-b779-25a03c9bb481","Type":"ContainerDied","Data":"e8f1f16c08ae3b9d460d7f044e732d0ad251cb2c4468a6ee45119a4c707a5076"}
Jul 16 22:57:06.424599 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.424418    2564 scope.go:117] "RemoveContainer" containerID="2414f2ac4619f8248c15570d1d989b5a259b50c39036c11b052214f600e20268"
Jul 16 22:57:06.446484 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.446456    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"]
Jul 16 22:57:06.451425 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.451399    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-c24jb"]
Jul 16 22:57:06.741055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.741019    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"]
Jul 16 22:57:06.741269 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:06.741243    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy" containerID="cri-o://879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795" gracePeriod=30
Jul 16 22:57:07.602041 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:07.602007    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" path="/var/lib/kubelet/pods/0ae968fd-2497-4aaa-b779-25a03c9bb481/volumes"
Jul 16 22:57:09.384586 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:09.384548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:57:11.991686 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:11.991663    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:57:12.068567 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068529    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z96cg\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-kube-api-access-z96cg\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068567 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068569    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-certs\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068785 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068601    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e082596-ea6e-472b-84d5-1412f4897d74-istio-podinfo\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068785 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e082596-ea6e-472b-84d5-1412f4897d74-istiod-ca-cert\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068785 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068721    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-istio-token\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068785 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068763    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-envoy\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068790    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-data\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068817    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-credential-socket\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068846    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-socket\") pod \"6e082596-ea6e-472b-84d5-1412f4897d74\" (UID: \"6e082596-ea6e-472b-84d5-1412f4897d74\") "
Jul 16 22:57:12.068996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.068907    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:57:12.069217 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.069128    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:57:12.069271 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.069230    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.069271 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.069260    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.069360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.069224    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e082596-ea6e-472b-84d5-1412f4897d74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:12.069360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.069325    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:57:12.069683 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.069643    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:57:12.070992 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.070970    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:57:12.071329 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.071305    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e082596-ea6e-472b-84d5-1412f4897d74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 22:57:12.071417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.071347    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:57:12.071479 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.071446    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-kube-api-access-z96cg" (OuterVolumeSpecName: "kube-api-access-z96cg") pod "6e082596-ea6e-472b-84d5-1412f4897d74" (UID: "6e082596-ea6e-472b-84d5-1412f4897d74"). InnerVolumeSpecName "kube-api-access-z96cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:57:12.170367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170332    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e082596-ea6e-472b-84d5-1412f4897d74-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.170367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170361    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.170367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170370    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.170367 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170377    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.170641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170386    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e082596-ea6e-472b-84d5-1412f4897d74-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.170641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170394    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z96cg\" (UniqueName: \"kubernetes.io/projected/6e082596-ea6e-472b-84d5-1412f4897d74-kube-api-access-z96cg\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.170641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.170402    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e082596-ea6e-472b-84d5-1412f4897d74-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.459571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.459536    2564 generic.go:356] "Generic (PLEG): container finished" podID="6e082596-ea6e-472b-84d5-1412f4897d74" containerID="879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795" exitCode=0
Jul 16 22:57:12.459740 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.459584    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"
Jul 16 22:57:12.459740 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.459619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" event={"ID":"6e082596-ea6e-472b-84d5-1412f4897d74","Type":"ContainerDied","Data":"879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795"}
Jul 16 22:57:12.459740 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.459677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm" event={"ID":"6e082596-ea6e-472b-84d5-1412f4897d74","Type":"ContainerDied","Data":"e212f850a0ec86a81d8a7255f30f3c449d6f4eed7d30c63a367a5cfaba8a4e20"}
Jul 16 22:57:12.459740 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.459703    2564 scope.go:117] "RemoveContainer" containerID="879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795"
Jul 16 22:57:12.469360 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.469344    2564 scope.go:117] "RemoveContainer" containerID="8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"
Jul 16 22:57:12.477432 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.477417    2564 scope.go:117] "RemoveContainer" containerID="879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795"
Jul 16 22:57:12.477701 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:12.477681    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795\": container with ID starting with 879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795 not found: ID does not exist" containerID="879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795"
Jul 16 22:57:12.477769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.477708    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795"} err="failed to get container status \"879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795\": rpc error: code = NotFound desc = could not find container \"879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795\": container with ID starting with 879ce00fd4da192c54ef32d14bf20dbf7e5721d841cafb20560c66f8035b7795 not found: ID does not exist"
Jul 16 22:57:12.477769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.477728    2564 scope.go:117] "RemoveContainer" containerID="8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"
Jul 16 22:57:12.477958 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:12.477942    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114\": container with ID starting with 8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114 not found: ID does not exist" containerID="8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"
Jul 16 22:57:12.478015 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.477967    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114"} err="failed to get container status \"8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114\": rpc error: code = NotFound desc = could not find container \"8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114\": container with ID starting with 8b0724b0b26a7b15e500070f7ae7561969416258253d72685c1efe5fb73e5114 not found: ID does not exist"
Jul 16 22:57:12.483336 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.483315    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"]
Jul 16 22:57:12.486326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:12.486305    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-75e8dc20-openshift-default-99cf95fff-4nvzm"]
Jul 16 22:57:13.602058 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:13.602023    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" path="/var/lib/kubelet/pods/6e082596-ea6e-472b-84d5-1412f4897d74/volumes"
Jul 16 22:57:20.097917 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.097879    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"]
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098234    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098247    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098259    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098264    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098272    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098277    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098284    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098289    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098301    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098306    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098312    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098317    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098327    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098332    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098380    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098387    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098394    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098399    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="847a482f-f435-48e2-b6c6-07de8a3ef219" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098406    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:20.098420 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.098411    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:20.102899 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.102882    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:20.105257 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.105236    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:57:20.105355 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.105270    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:57:20.109497 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.109474    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"]
Jul 16 22:57:20.243943 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.243910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f30bcaf8-21ef-4955-9e8a-ef3785ae0366-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl\" (UID: \"f30bcaf8-21ef-4955-9e8a-ef3785ae0366\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:20.345049 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.345020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f30bcaf8-21ef-4955-9e8a-ef3785ae0366-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl\" (UID: \"f30bcaf8-21ef-4955-9e8a-ef3785ae0366\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:20.345659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.345636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f30bcaf8-21ef-4955-9e8a-ef3785ae0366-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl\" (UID: \"f30bcaf8-21ef-4955-9e8a-ef3785ae0366\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:20.414756 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.414655    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:20.549250 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:20.549223    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"]
Jul 16 22:57:20.550899 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:57:20.550872    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30bcaf8_21ef_4955_9e8a_ef3785ae0366.slice/crio-72292b2e2db5de610ea84dacfe100e752be27a57eba1d03fbcf2486b7a177d94 WatchSource:0}: Error finding container 72292b2e2db5de610ea84dacfe100e752be27a57eba1d03fbcf2486b7a177d94: Status 404 returned error can't find the container with id 72292b2e2db5de610ea84dacfe100e752be27a57eba1d03fbcf2486b7a177d94
Jul 16 22:57:21.509930 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:21.509892    2564 generic.go:356] "Generic (PLEG): container finished" podID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerID="9c9204b90910bf5c94aaba952fc0d10098a92b19b9ea066e367aac660c99a1ef" exitCode=6
Jul 16 22:57:21.510373 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:21.509984    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" event={"ID":"f30bcaf8-21ef-4955-9e8a-ef3785ae0366","Type":"ContainerDied","Data":"9c9204b90910bf5c94aaba952fc0d10098a92b19b9ea066e367aac660c99a1ef"}
Jul 16 22:57:21.510373 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:21.510057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" event={"ID":"f30bcaf8-21ef-4955-9e8a-ef3785ae0366","Type":"ContainerStarted","Data":"72292b2e2db5de610ea84dacfe100e752be27a57eba1d03fbcf2486b7a177d94"}
Jul 16 22:57:21.510373 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:21.510186    2564 scope.go:117] "RemoveContainer" containerID="9c9204b90910bf5c94aaba952fc0d10098a92b19b9ea066e367aac660c99a1ef"
Jul 16 22:57:22.104045 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104013    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"]
Jul 16 22:57:22.104374 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104361    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:22.104428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104376    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:22.104428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104388    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:22.104428 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104394    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:22.104523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104446    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="961c8d8b-2570-47f0-8308-d69a07226f55" containerName="revoke-keys"
Jul 16 22:57:22.104523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104456    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e082596-ea6e-472b-84d5-1412f4897d74" containerName="istio-proxy"
Jul 16 22:57:22.104523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.104463    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae968fd-2497-4aaa-b779-25a03c9bb481" containerName="revoke-keys"
Jul 16 22:57:22.107281 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.107264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:22.116351 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.116326    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"]
Jul 16 22:57:22.262866 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.262828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4480d765-61c3-4d0f-bc2a-6bd73046305e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h\" (UID: \"4480d765-61c3-4d0f-bc2a-6bd73046305e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:22.364087 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.363995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4480d765-61c3-4d0f-bc2a-6bd73046305e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h\" (UID: \"4480d765-61c3-4d0f-bc2a-6bd73046305e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:22.364655 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.364630    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4480d765-61c3-4d0f-bc2a-6bd73046305e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h\" (UID: \"4480d765-61c3-4d0f-bc2a-6bd73046305e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:22.432974 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.432928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:22.517917 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.517599    2564 generic.go:356] "Generic (PLEG): container finished" podID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerID="f46d1f5731af4dd3626ac177e59b92b68075c2aae0a8383c5564ebd9918bc051" exitCode=6
Jul 16 22:57:22.517917 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.517747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" event={"ID":"f30bcaf8-21ef-4955-9e8a-ef3785ae0366","Type":"ContainerDied","Data":"f46d1f5731af4dd3626ac177e59b92b68075c2aae0a8383c5564ebd9918bc051"}
Jul 16 22:57:22.517917 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.517805    2564 scope.go:117] "RemoveContainer" containerID="9c9204b90910bf5c94aaba952fc0d10098a92b19b9ea066e367aac660c99a1ef"
Jul 16 22:57:22.518703 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.518115    2564 scope.go:117] "RemoveContainer" containerID="f46d1f5731af4dd3626ac177e59b92b68075c2aae0a8383c5564ebd9918bc051"
Jul 16 22:57:22.518703 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:22.518404    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl_odh-ai-gateway-infra(f30bcaf8-21ef-4955-9e8a-ef3785ae0366)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366"
Jul 16 22:57:22.569422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:22.569401    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"]
Jul 16 22:57:22.571399 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:57:22.571367    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4480d765_61c3_4d0f_bc2a_6bd73046305e.slice/crio-72b9592e3593a9d1dcaa4f27932f369544b32de11b7bf91a1278e414378f278a WatchSource:0}: Error finding container 72b9592e3593a9d1dcaa4f27932f369544b32de11b7bf91a1278e414378f278a: Status 404 returned error can't find the container with id 72b9592e3593a9d1dcaa4f27932f369544b32de11b7bf91a1278e414378f278a
Jul 16 22:57:23.525968 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:23.525939    2564 scope.go:117] "RemoveContainer" containerID="f46d1f5731af4dd3626ac177e59b92b68075c2aae0a8383c5564ebd9918bc051"
Jul 16 22:57:23.526424 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:23.526184    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl_odh-ai-gateway-infra(f30bcaf8-21ef-4955-9e8a-ef3785ae0366)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366"
Jul 16 22:57:23.527219 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:23.527194    2564 generic.go:356] "Generic (PLEG): container finished" podID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerID="e0484a51410aa676535d7696d6b73dfa97442e01d8f2fc04ab673277fd13ba73" exitCode=6
Jul 16 22:57:23.527323 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:23.527254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" event={"ID":"4480d765-61c3-4d0f-bc2a-6bd73046305e","Type":"ContainerDied","Data":"e0484a51410aa676535d7696d6b73dfa97442e01d8f2fc04ab673277fd13ba73"}
Jul 16 22:57:23.527323 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:23.527305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" event={"ID":"4480d765-61c3-4d0f-bc2a-6bd73046305e","Type":"ContainerStarted","Data":"72b9592e3593a9d1dcaa4f27932f369544b32de11b7bf91a1278e414378f278a"}
Jul 16 22:57:23.527464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:23.527452    2564 scope.go:117] "RemoveContainer" containerID="e0484a51410aa676535d7696d6b73dfa97442e01d8f2fc04ab673277fd13ba73"
Jul 16 22:57:24.533864 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:24.533826    2564 generic.go:356] "Generic (PLEG): container finished" podID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerID="0e014326ae181918810f434f6dd53066e88fcf1b348c6187e091b5d595d32ede" exitCode=6
Jul 16 22:57:24.534384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:24.533898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" event={"ID":"4480d765-61c3-4d0f-bc2a-6bd73046305e","Type":"ContainerDied","Data":"0e014326ae181918810f434f6dd53066e88fcf1b348c6187e091b5d595d32ede"}
Jul 16 22:57:24.534384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:24.534006    2564 scope.go:117] "RemoveContainer" containerID="e0484a51410aa676535d7696d6b73dfa97442e01d8f2fc04ab673277fd13ba73"
Jul 16 22:57:24.534384 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:24.534226    2564 scope.go:117] "RemoveContainer" containerID="0e014326ae181918810f434f6dd53066e88fcf1b348c6187e091b5d595d32ede"
Jul 16 22:57:24.534556 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:24.534472    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h_odh-ai-gateway-infra(4480d765-61c3-4d0f-bc2a-6bd73046305e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e"
Jul 16 22:57:24.546408 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:24.546380    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:57:24.557878 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:24.557853    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 22:57:25.540643 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:25.540616    2564 scope.go:117] "RemoveContainer" containerID="0e014326ae181918810f434f6dd53066e88fcf1b348c6187e091b5d595d32ede"
Jul 16 22:57:25.541034 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:25.540825    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h_odh-ai-gateway-infra(4480d765-61c3-4d0f-bc2a-6bd73046305e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e"
Jul 16 22:57:28.664650 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.664616    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"]
Jul 16 22:57:28.669417 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.669398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:28.676671 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.676638    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"]
Jul 16 22:57:28.822949 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.822914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/952818f6-9a24-4803-8ca2-7e44a8fead1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-h2j2g\" (UID: \"952818f6-9a24-4803-8ca2-7e44a8fead1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:28.923926 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.923824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/952818f6-9a24-4803-8ca2-7e44a8fead1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-h2j2g\" (UID: \"952818f6-9a24-4803-8ca2-7e44a8fead1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:28.924502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.924483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/952818f6-9a24-4803-8ca2-7e44a8fead1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-h2j2g\" (UID: \"952818f6-9a24-4803-8ca2-7e44a8fead1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:28.981206 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:28.981175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:29.114291 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:29.114265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"]
Jul 16 22:57:29.116515 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:57:29.116485    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952818f6_9a24_4803_8ca2_7e44a8fead1e.slice/crio-0afb71db262f156f79d716095d5ccc02dd23f0028c1ad2a57229abc9cc85aca7 WatchSource:0}: Error finding container 0afb71db262f156f79d716095d5ccc02dd23f0028c1ad2a57229abc9cc85aca7: Status 404 returned error can't find the container with id 0afb71db262f156f79d716095d5ccc02dd23f0028c1ad2a57229abc9cc85aca7
Jul 16 22:57:29.562760 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:29.562718    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" event={"ID":"952818f6-9a24-4803-8ca2-7e44a8fead1e","Type":"ContainerStarted","Data":"f98ffb7537ae5f2d4723e1ad17c403e5d193d66fa46b8d0a94b36365fcb0910b"}
Jul 16 22:57:29.562760 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:29.562768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" event={"ID":"952818f6-9a24-4803-8ca2-7e44a8fead1e","Type":"ContainerStarted","Data":"0afb71db262f156f79d716095d5ccc02dd23f0028c1ad2a57229abc9cc85aca7"}
Jul 16 22:57:29.586958 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:29.586903    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" podStartSLOduration=1.586884967 podStartE2EDuration="1.586884967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:57:29.586116745 +0000 UTC m=+2405.906672818" watchObservedRunningTime="2026-07-16 22:57:29.586884967 +0000 UTC m=+2405.907441064"
Jul 16 22:57:30.569600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:30.569567    2564 generic.go:356] "Generic (PLEG): container finished" podID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerID="f98ffb7537ae5f2d4723e1ad17c403e5d193d66fa46b8d0a94b36365fcb0910b" exitCode=6
Jul 16 22:57:30.570090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:30.569614    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" event={"ID":"952818f6-9a24-4803-8ca2-7e44a8fead1e","Type":"ContainerDied","Data":"f98ffb7537ae5f2d4723e1ad17c403e5d193d66fa46b8d0a94b36365fcb0910b"}
Jul 16 22:57:30.570090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:30.569829    2564 scope.go:117] "RemoveContainer" containerID="f98ffb7537ae5f2d4723e1ad17c403e5d193d66fa46b8d0a94b36365fcb0910b"
Jul 16 22:57:31.576608 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:31.576574    2564 generic.go:356] "Generic (PLEG): container finished" podID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerID="8226b919891ac2b933a7141b006370cf52c2703079b2a4711c439b3710540299" exitCode=6
Jul 16 22:57:31.577044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:31.576638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" event={"ID":"952818f6-9a24-4803-8ca2-7e44a8fead1e","Type":"ContainerDied","Data":"8226b919891ac2b933a7141b006370cf52c2703079b2a4711c439b3710540299"}
Jul 16 22:57:31.577044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:31.576693    2564 scope.go:117] "RemoveContainer" containerID="f98ffb7537ae5f2d4723e1ad17c403e5d193d66fa46b8d0a94b36365fcb0910b"
Jul 16 22:57:31.577044 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:31.576871    2564 scope.go:117] "RemoveContainer" containerID="8226b919891ac2b933a7141b006370cf52c2703079b2a4711c439b3710540299"
Jul 16 22:57:31.577197 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:31.577080    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-h2j2g_odh-ai-gateway-infra(952818f6-9a24-4803-8ca2-7e44a8fead1e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e"
Jul 16 22:57:32.583090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:32.583063    2564 scope.go:117] "RemoveContainer" containerID="8226b919891ac2b933a7141b006370cf52c2703079b2a4711c439b3710540299"
Jul 16 22:57:32.583486 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:32.583291    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-h2j2g_odh-ai-gateway-infra(952818f6-9a24-4803-8ca2-7e44a8fead1e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e"
Jul 16 22:57:35.603423 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:35.603393    2564 scope.go:117] "RemoveContainer" containerID="f46d1f5731af4dd3626ac177e59b92b68075c2aae0a8383c5564ebd9918bc051"
Jul 16 22:57:36.607210 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.607172    2564 generic.go:356] "Generic (PLEG): container finished" podID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerID="bc2bede89f200a42198e9c88387d255984e6b08c02add7244b7f126465cbda56" exitCode=6
Jul 16 22:57:36.607610 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.607258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" event={"ID":"f30bcaf8-21ef-4955-9e8a-ef3785ae0366","Type":"ContainerDied","Data":"bc2bede89f200a42198e9c88387d255984e6b08c02add7244b7f126465cbda56"}
Jul 16 22:57:36.607610 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.607325    2564 scope.go:117] "RemoveContainer" containerID="f46d1f5731af4dd3626ac177e59b92b68075c2aae0a8383c5564ebd9918bc051"
Jul 16 22:57:36.607610 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.607580    2564 scope.go:117] "RemoveContainer" containerID="bc2bede89f200a42198e9c88387d255984e6b08c02add7244b7f126465cbda56"
Jul 16 22:57:36.607802 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:36.607783    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl_odh-ai-gateway-infra(f30bcaf8-21ef-4955-9e8a-ef3785ae0366)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366"
Jul 16 22:57:36.810190 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.810152    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"]
Jul 16 22:57:36.814755 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.814738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:36.824675 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.824647    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"]
Jul 16 22:57:36.887334 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.887243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81c78ce-2629-409b-a1ad-92d08dff1d28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2\" (UID: \"f81c78ce-2629-409b-a1ad-92d08dff1d28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:36.988517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.988481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81c78ce-2629-409b-a1ad-92d08dff1d28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2\" (UID: \"f81c78ce-2629-409b-a1ad-92d08dff1d28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:36.989101 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:36.989083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81c78ce-2629-409b-a1ad-92d08dff1d28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2\" (UID: \"f81c78ce-2629-409b-a1ad-92d08dff1d28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:37.125719 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.125681    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:37.256220 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.256196    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"]
Jul 16 22:57:37.257676 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:57:37.257651    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81c78ce_2629_409b_a1ad_92d08dff1d28.slice/crio-6749a26debd0f8cca727d1578a5e2a1247f87dea332418da485c836b44133866 WatchSource:0}: Error finding container 6749a26debd0f8cca727d1578a5e2a1247f87dea332418da485c836b44133866: Status 404 returned error can't find the container with id 6749a26debd0f8cca727d1578a5e2a1247f87dea332418da485c836b44133866
Jul 16 22:57:37.613530 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.613492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" event={"ID":"f81c78ce-2629-409b-a1ad-92d08dff1d28","Type":"ContainerStarted","Data":"b72a61268deaabe1a49a87697fdd6809f714f90f1791b032c46bd544531174e0"}
Jul 16 22:57:37.614016 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.613552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" event={"ID":"f81c78ce-2629-409b-a1ad-92d08dff1d28","Type":"ContainerStarted","Data":"6749a26debd0f8cca727d1578a5e2a1247f87dea332418da485c836b44133866"}
Jul 16 22:57:37.631724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.631680    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" podStartSLOduration=1.631664968 podStartE2EDuration="1.631664968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:57:37.628958657 +0000 UTC m=+2413.949514739" watchObservedRunningTime="2026-07-16 22:57:37.631664968 +0000 UTC m=+2413.952221103"
Jul 16 22:57:37.635458 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.635433    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"]
Jul 16 22:57:37.791215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.791189    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:37.800183 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.800165    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f30bcaf8-21ef-4955-9e8a-ef3785ae0366-maas-api-service-ca\") pod \"f30bcaf8-21ef-4955-9e8a-ef3785ae0366\" (UID: \"f30bcaf8-21ef-4955-9e8a-ef3785ae0366\") "
Jul 16 22:57:37.800522 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.800503    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f30bcaf8-21ef-4955-9e8a-ef3785ae0366-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f30bcaf8-21ef-4955-9e8a-ef3785ae0366" (UID: "f30bcaf8-21ef-4955-9e8a-ef3785ae0366"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:37.901481 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:37.901446    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f30bcaf8-21ef-4955-9e8a-ef3785ae0366-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:38.621624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.621597    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"
Jul 16 22:57:38.622121 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.621640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl" event={"ID":"f30bcaf8-21ef-4955-9e8a-ef3785ae0366","Type":"ContainerDied","Data":"72292b2e2db5de610ea84dacfe100e752be27a57eba1d03fbcf2486b7a177d94"}
Jul 16 22:57:38.622121 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.621717    2564 scope.go:117] "RemoveContainer" containerID="bc2bede89f200a42198e9c88387d255984e6b08c02add7244b7f126465cbda56"
Jul 16 22:57:38.623239 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.623211    2564 generic.go:356] "Generic (PLEG): container finished" podID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerID="b72a61268deaabe1a49a87697fdd6809f714f90f1791b032c46bd544531174e0" exitCode=6
Jul 16 22:57:38.623364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.623259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" event={"ID":"f81c78ce-2629-409b-a1ad-92d08dff1d28","Type":"ContainerDied","Data":"b72a61268deaabe1a49a87697fdd6809f714f90f1791b032c46bd544531174e0"}
Jul 16 22:57:38.623508 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.623493    2564 scope.go:117] "RemoveContainer" containerID="b72a61268deaabe1a49a87697fdd6809f714f90f1791b032c46bd544531174e0"
Jul 16 22:57:38.657305 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.657281    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"]
Jul 16 22:57:38.662992 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:38.662966    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-7qrjl"]
Jul 16 22:57:39.602209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:39.602178    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" path="/var/lib/kubelet/pods/f30bcaf8-21ef-4955-9e8a-ef3785ae0366/volumes"
Jul 16 22:57:39.636831 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:39.636792    2564 generic.go:356] "Generic (PLEG): container finished" podID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerID="12379b04cd2a480d14a63c406c5deab5ff00bbeedc5da9ff739bd0dcc91a04e6" exitCode=6
Jul 16 22:57:39.637354 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:39.636838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" event={"ID":"f81c78ce-2629-409b-a1ad-92d08dff1d28","Type":"ContainerDied","Data":"12379b04cd2a480d14a63c406c5deab5ff00bbeedc5da9ff739bd0dcc91a04e6"}
Jul 16 22:57:39.637354 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:39.636908    2564 scope.go:117] "RemoveContainer" containerID="b72a61268deaabe1a49a87697fdd6809f714f90f1791b032c46bd544531174e0"
Jul 16 22:57:39.637354 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:39.637094    2564 scope.go:117] "RemoveContainer" containerID="12379b04cd2a480d14a63c406c5deab5ff00bbeedc5da9ff739bd0dcc91a04e6"
Jul 16 22:57:39.637354 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:39.637342    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2_odh-ai-gateway-infra(f81c78ce-2629-409b-a1ad-92d08dff1d28)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28"
Jul 16 22:57:40.596835 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:40.596800    2564 scope.go:117] "RemoveContainer" containerID="0e014326ae181918810f434f6dd53066e88fcf1b348c6187e091b5d595d32ede"
Jul 16 22:57:40.643152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:40.643112    2564 scope.go:117] "RemoveContainer" containerID="12379b04cd2a480d14a63c406c5deab5ff00bbeedc5da9ff739bd0dcc91a04e6"
Jul 16 22:57:40.643534 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:40.643314    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2_odh-ai-gateway-infra(f81c78ce-2629-409b-a1ad-92d08dff1d28)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28"
Jul 16 22:57:41.649978 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:41.649947    2564 generic.go:356] "Generic (PLEG): container finished" podID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerID="3d1e781debf48e708b8a11f893aabe9a8b732dbe56dedf2398f97b557cee8956" exitCode=6
Jul 16 22:57:41.650374 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:41.650031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" event={"ID":"4480d765-61c3-4d0f-bc2a-6bd73046305e","Type":"ContainerDied","Data":"3d1e781debf48e708b8a11f893aabe9a8b732dbe56dedf2398f97b557cee8956"}
Jul 16 22:57:41.650374 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:41.650099    2564 scope.go:117] "RemoveContainer" containerID="0e014326ae181918810f434f6dd53066e88fcf1b348c6187e091b5d595d32ede"
Jul 16 22:57:41.650472 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:41.650427    2564 scope.go:117] "RemoveContainer" containerID="3d1e781debf48e708b8a11f893aabe9a8b732dbe56dedf2398f97b557cee8956"
Jul 16 22:57:41.650655 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:41.650636    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h_odh-ai-gateway-infra(4480d765-61c3-4d0f-bc2a-6bd73046305e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e"
Jul 16 22:57:42.678391 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:42.678362    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"]
Jul 16 22:57:42.811892 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:42.811868    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:42.842676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:42.842645    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4480d765-61c3-4d0f-bc2a-6bd73046305e-maas-api-service-ca\") pod \"4480d765-61c3-4d0f-bc2a-6bd73046305e\" (UID: \"4480d765-61c3-4d0f-bc2a-6bd73046305e\") "
Jul 16 22:57:42.843035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:42.843009    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4480d765-61c3-4d0f-bc2a-6bd73046305e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4480d765-61c3-4d0f-bc2a-6bd73046305e" (UID: "4480d765-61c3-4d0f-bc2a-6bd73046305e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:42.944175 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:42.944064    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4480d765-61c3-4d0f-bc2a-6bd73046305e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:43.663177 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:43.663149    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"
Jul 16 22:57:43.663177 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:43.663162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h" event={"ID":"4480d765-61c3-4d0f-bc2a-6bd73046305e","Type":"ContainerDied","Data":"72b9592e3593a9d1dcaa4f27932f369544b32de11b7bf91a1278e414378f278a"}
Jul 16 22:57:43.663405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:43.663222    2564 scope.go:117] "RemoveContainer" containerID="3d1e781debf48e708b8a11f893aabe9a8b732dbe56dedf2398f97b557cee8956"
Jul 16 22:57:43.680051 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:43.680026    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"]
Jul 16 22:57:43.684527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:43.684505    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tf22h"]
Jul 16 22:57:44.596049 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:44.596014    2564 scope.go:117] "RemoveContainer" containerID="8226b919891ac2b933a7141b006370cf52c2703079b2a4711c439b3710540299"
Jul 16 22:57:45.602790 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:45.602663    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" path="/var/lib/kubelet/pods/4480d765-61c3-4d0f-bc2a-6bd73046305e/volumes"
Jul 16 22:57:45.676049 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:45.676020    2564 generic.go:356] "Generic (PLEG): container finished" podID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerID="1e9764559c23ba7291fea80d52b11ce4dd7bb2797a5b215313c60dfded98a04c" exitCode=6
Jul 16 22:57:45.676241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:45.676098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" event={"ID":"952818f6-9a24-4803-8ca2-7e44a8fead1e","Type":"ContainerDied","Data":"1e9764559c23ba7291fea80d52b11ce4dd7bb2797a5b215313c60dfded98a04c"}
Jul 16 22:57:45.676241 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:45.676174    2564 scope.go:117] "RemoveContainer" containerID="8226b919891ac2b933a7141b006370cf52c2703079b2a4711c439b3710540299"
Jul 16 22:57:45.676405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:45.676391    2564 scope.go:117] "RemoveContainer" containerID="1e9764559c23ba7291fea80d52b11ce4dd7bb2797a5b215313c60dfded98a04c"
Jul 16 22:57:45.676605 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:45.676587    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-h2j2g_odh-ai-gateway-infra(952818f6-9a24-4803-8ca2-7e44a8fead1e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e"
Jul 16 22:57:46.702710 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:46.702677    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"]
Jul 16 22:57:46.837458 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:46.837435    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:46.880943 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:46.880913    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/952818f6-9a24-4803-8ca2-7e44a8fead1e-maas-api-service-ca\") pod \"952818f6-9a24-4803-8ca2-7e44a8fead1e\" (UID: \"952818f6-9a24-4803-8ca2-7e44a8fead1e\") "
Jul 16 22:57:46.881254 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:46.881234    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952818f6-9a24-4803-8ca2-7e44a8fead1e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "952818f6-9a24-4803-8ca2-7e44a8fead1e" (UID: "952818f6-9a24-4803-8ca2-7e44a8fead1e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:46.982532 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:46.982446    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/952818f6-9a24-4803-8ca2-7e44a8fead1e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:47.688862 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:47.688831    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"
Jul 16 22:57:47.689006 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:47.688880    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g" event={"ID":"952818f6-9a24-4803-8ca2-7e44a8fead1e","Type":"ContainerDied","Data":"0afb71db262f156f79d716095d5ccc02dd23f0028c1ad2a57229abc9cc85aca7"}
Jul 16 22:57:47.689006 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:47.688938    2564 scope.go:117] "RemoveContainer" containerID="1e9764559c23ba7291fea80d52b11ce4dd7bb2797a5b215313c60dfded98a04c"
Jul 16 22:57:47.706173 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:47.706144    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"]
Jul 16 22:57:47.710152 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:47.710112    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-h2j2g"]
Jul 16 22:57:49.601361 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:49.601325    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" path="/var/lib/kubelet/pods/952818f6-9a24-4803-8ca2-7e44a8fead1e/volumes"
Jul 16 22:57:53.597192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:53.597162    2564 scope.go:117] "RemoveContainer" containerID="12379b04cd2a480d14a63c406c5deab5ff00bbeedc5da9ff739bd0dcc91a04e6"
Jul 16 22:57:54.732832 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:54.732798    2564 generic.go:356] "Generic (PLEG): container finished" podID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerID="30978e2bd43b7a89e4ce80ed597fb6d9d2d34a407ac91995583c1bad602d8fa7" exitCode=6
Jul 16 22:57:54.733309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:54.732874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" event={"ID":"f81c78ce-2629-409b-a1ad-92d08dff1d28","Type":"ContainerDied","Data":"30978e2bd43b7a89e4ce80ed597fb6d9d2d34a407ac91995583c1bad602d8fa7"}
Jul 16 22:57:54.733309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:54.732937    2564 scope.go:117] "RemoveContainer" containerID="12379b04cd2a480d14a63c406c5deab5ff00bbeedc5da9ff739bd0dcc91a04e6"
Jul 16 22:57:54.733309 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:54.733245    2564 scope.go:117] "RemoveContainer" containerID="30978e2bd43b7a89e4ce80ed597fb6d9d2d34a407ac91995583c1bad602d8fa7"
Jul 16 22:57:54.733516 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:57:54.733495    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2_odh-ai-gateway-infra(f81c78ce-2629-409b-a1ad-92d08dff1d28)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28"
Jul 16 22:57:55.761205 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:55.761173    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"]
Jul 16 22:57:55.895884 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:55.895860    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:55.969527 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:55.969494    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81c78ce-2629-409b-a1ad-92d08dff1d28-maas-api-service-ca\") pod \"f81c78ce-2629-409b-a1ad-92d08dff1d28\" (UID: \"f81c78ce-2629-409b-a1ad-92d08dff1d28\") "
Jul 16 22:57:55.969846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:55.969825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f81c78ce-2629-409b-a1ad-92d08dff1d28-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f81c78ce-2629-409b-a1ad-92d08dff1d28" (UID: "f81c78ce-2629-409b-a1ad-92d08dff1d28"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:56.071004 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:56.070919    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f81c78ce-2629-409b-a1ad-92d08dff1d28-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:57:56.746397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:56.746368    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"
Jul 16 22:57:56.746397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:56.746376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2" event={"ID":"f81c78ce-2629-409b-a1ad-92d08dff1d28","Type":"ContainerDied","Data":"6749a26debd0f8cca727d1578a5e2a1247f87dea332418da485c836b44133866"}
Jul 16 22:57:56.746648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:56.746443    2564 scope.go:117] "RemoveContainer" containerID="30978e2bd43b7a89e4ce80ed597fb6d9d2d34a407ac91995583c1bad602d8fa7"
Jul 16 22:57:56.768952 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:56.768926    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"]
Jul 16 22:57:56.779533 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:56.779497    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfxc2"]
Jul 16 22:57:57.601460 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:57:57.601423    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" path="/var/lib/kubelet/pods/f81c78ce-2629-409b-a1ad-92d08dff1d28/volumes"
Jul 16 22:58:10.152243 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152207    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"]
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152538    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152550    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152558    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152566    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152575    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152582    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152592    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152598    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152607    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152613    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152618    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152623    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152634    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152640    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152649    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152656    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152662    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152667    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152673    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152678    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152741    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.152738 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152749    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:10.153426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152755    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.153426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152761    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:10.153426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152767    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:10.153426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152771    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:10.153426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152777    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:10.153426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.152781    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:10.155841 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.155823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:10.158480 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.158456    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:58:10.158621 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.158518    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:58:10.167491 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.167465    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"]
Jul 16 22:58:10.293763 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.293729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af33c376-b77e-43d9-8d0c-70067acc237d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l\" (UID: \"af33c376-b77e-43d9-8d0c-70067acc237d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:10.395328 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.395285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af33c376-b77e-43d9-8d0c-70067acc237d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l\" (UID: \"af33c376-b77e-43d9-8d0c-70067acc237d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:10.395875 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.395856    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af33c376-b77e-43d9-8d0c-70067acc237d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l\" (UID: \"af33c376-b77e-43d9-8d0c-70067acc237d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:10.465795 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.465765    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:10.600579 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.600557    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"]
Jul 16 22:58:10.602519 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:58:10.602491    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf33c376_b77e_43d9_8d0c_70067acc237d.slice/crio-8039ed01dc6a7fc1e731fe613b5adba618b1731f3843fb1a5149441286848690 WatchSource:0}: Error finding container 8039ed01dc6a7fc1e731fe613b5adba618b1731f3843fb1a5149441286848690: Status 404 returned error can't find the container with id 8039ed01dc6a7fc1e731fe613b5adba618b1731f3843fb1a5149441286848690
Jul 16 22:58:10.819422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.819387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerStarted","Data":"851c4ce23f5f29b7aa554de69200b7d8e3d401e83121e0947a84c528df9933d9"}
Jul 16 22:58:10.819576 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.819486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerStarted","Data":"8039ed01dc6a7fc1e731fe613b5adba618b1731f3843fb1a5149441286848690"}
Jul 16 22:58:10.838040 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:10.837982    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" podStartSLOduration=0.837967593 podStartE2EDuration="837.967593ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:58:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:58:10.835695127 +0000 UTC m=+2447.156251199" watchObservedRunningTime="2026-07-16 22:58:10.837967593 +0000 UTC m=+2447.158523664"
Jul 16 22:58:11.825788 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:11.825748    2564 generic.go:356] "Generic (PLEG): container finished" podID="af33c376-b77e-43d9-8d0c-70067acc237d" containerID="851c4ce23f5f29b7aa554de69200b7d8e3d401e83121e0947a84c528df9933d9" exitCode=6
Jul 16 22:58:11.826209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:11.825830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerDied","Data":"851c4ce23f5f29b7aa554de69200b7d8e3d401e83121e0947a84c528df9933d9"}
Jul 16 22:58:11.826209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:11.826023    2564 scope.go:117] "RemoveContainer" containerID="851c4ce23f5f29b7aa554de69200b7d8e3d401e83121e0947a84c528df9933d9"
Jul 16 22:58:12.833812 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:12.833776    2564 generic.go:356] "Generic (PLEG): container finished" podID="af33c376-b77e-43d9-8d0c-70067acc237d" containerID="1b6a4f6b1f6567ae1c19c3d7a5792dbe402e32b17b6c519c3113bf31983362df" exitCode=6
Jul 16 22:58:12.834296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:12.833854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerDied","Data":"1b6a4f6b1f6567ae1c19c3d7a5792dbe402e32b17b6c519c3113bf31983362df"}
Jul 16 22:58:12.834296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:12.833906    2564 scope.go:117] "RemoveContainer" containerID="851c4ce23f5f29b7aa554de69200b7d8e3d401e83121e0947a84c528df9933d9"
Jul 16 22:58:12.834296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:12.834053    2564 scope.go:117] "RemoveContainer" containerID="1b6a4f6b1f6567ae1c19c3d7a5792dbe402e32b17b6c519c3113bf31983362df"
Jul 16 22:58:12.834437 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:12.834303    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l_odh-ai-gateway-infra(af33c376-b77e-43d9-8d0c-70067acc237d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" podUID="af33c376-b77e-43d9-8d0c-70067acc237d"
Jul 16 22:58:13.840112 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:13.840085    2564 scope.go:117] "RemoveContainer" containerID="1b6a4f6b1f6567ae1c19c3d7a5792dbe402e32b17b6c519c3113bf31983362df"
Jul 16 22:58:13.840529 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:13.840339    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l_odh-ai-gateway-infra(af33c376-b77e-43d9-8d0c-70067acc237d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" podUID="af33c376-b77e-43d9-8d0c-70067acc237d"
Jul 16 22:58:18.709008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.708972    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"]
Jul 16 22:58:18.709421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709330    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:18.709421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709342    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:18.709421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709357    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:18.709421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709363    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:18.709421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709410    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4480d765-61c3-4d0f-bc2a-6bd73046305e" containerName="revoke-keys"
Jul 16 22:58:18.709421 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709420    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="952818f6-9a24-4803-8ca2-7e44a8fead1e" containerName="revoke-keys"
Jul 16 22:58:18.709624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709428    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f30bcaf8-21ef-4955-9e8a-ef3785ae0366" containerName="revoke-keys"
Jul 16 22:58:18.709624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.709435    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81c78ce-2629-409b-a1ad-92d08dff1d28" containerName="revoke-keys"
Jul 16 22:58:18.713769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.713748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:18.720726 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.720696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"]
Jul 16 22:58:18.867728 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.867685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d877c6f-6c3f-4d32-9bbd-4860b07158be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-qvwxs\" (UID: \"0d877c6f-6c3f-4d32-9bbd-4860b07158be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:18.969220 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.969082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d877c6f-6c3f-4d32-9bbd-4860b07158be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-qvwxs\" (UID: \"0d877c6f-6c3f-4d32-9bbd-4860b07158be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:18.969742 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:18.969719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d877c6f-6c3f-4d32-9bbd-4860b07158be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-qvwxs\" (UID: \"0d877c6f-6c3f-4d32-9bbd-4860b07158be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:19.025463 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:19.025430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:19.367783 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:19.367759    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"]
Jul 16 22:58:19.369659 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:58:19.369631    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d877c6f_6c3f_4d32_9bbd_4860b07158be.slice/crio-58e323ae6ec23e36f7e29cf7e132b2e2cc1d8ac43f93fb0d7764432e62b8f1bb WatchSource:0}: Error finding container 58e323ae6ec23e36f7e29cf7e132b2e2cc1d8ac43f93fb0d7764432e62b8f1bb: Status 404 returned error can't find the container with id 58e323ae6ec23e36f7e29cf7e132b2e2cc1d8ac43f93fb0d7764432e62b8f1bb
Jul 16 22:58:19.872168 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:19.872115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerStarted","Data":"69335d44a41ad474ad7bd7932d25a21ae30f8d868a9e6197e1e6cb0e0a81a596"}
Jul 16 22:58:19.872546 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:19.872228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerStarted","Data":"58e323ae6ec23e36f7e29cf7e132b2e2cc1d8ac43f93fb0d7764432e62b8f1bb"}
Jul 16 22:58:19.889684 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:19.889625    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" podStartSLOduration=1.889607534 podStartE2EDuration="1.889607534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:58:19.888542802 +0000 UTC m=+2456.209098875" watchObservedRunningTime="2026-07-16 22:58:19.889607534 +0000 UTC m=+2456.210163605"
Jul 16 22:58:20.879326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:20.879288    2564 generic.go:356] "Generic (PLEG): container finished" podID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerID="69335d44a41ad474ad7bd7932d25a21ae30f8d868a9e6197e1e6cb0e0a81a596" exitCode=6
Jul 16 22:58:20.879752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:20.879375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerDied","Data":"69335d44a41ad474ad7bd7932d25a21ae30f8d868a9e6197e1e6cb0e0a81a596"}
Jul 16 22:58:20.879752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:20.879583    2564 scope.go:117] "RemoveContainer" containerID="69335d44a41ad474ad7bd7932d25a21ae30f8d868a9e6197e1e6cb0e0a81a596"
Jul 16 22:58:21.886710 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:21.886607    2564 generic.go:356] "Generic (PLEG): container finished" podID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerID="3a8965b1a3eede82dc72b65b3bc8562de7c1761fadbb27480297778243cd4535" exitCode=6
Jul 16 22:58:21.887095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:21.886701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerDied","Data":"3a8965b1a3eede82dc72b65b3bc8562de7c1761fadbb27480297778243cd4535"}
Jul 16 22:58:21.887095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:21.886764    2564 scope.go:117] "RemoveContainer" containerID="69335d44a41ad474ad7bd7932d25a21ae30f8d868a9e6197e1e6cb0e0a81a596"
Jul 16 22:58:21.887095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:21.886875    2564 scope.go:117] "RemoveContainer" containerID="3a8965b1a3eede82dc72b65b3bc8562de7c1761fadbb27480297778243cd4535"
Jul 16 22:58:21.887267 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:21.887103    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-qvwxs_odh-ai-gateway-infra(0d877c6f-6c3f-4d32-9bbd-4860b07158be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be"
Jul 16 22:58:22.183019 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.182938    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"]
Jul 16 22:58:22.187400 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.187379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:22.196620 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.196598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94acfba-1f89-4509-9acf-4977b3f100f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x\" (UID: \"a94acfba-1f89-4509-9acf-4977b3f100f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:22.210197 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.210163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"]
Jul 16 22:58:22.297090 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.297054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94acfba-1f89-4509-9acf-4977b3f100f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x\" (UID: \"a94acfba-1f89-4509-9acf-4977b3f100f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:22.297692 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.297672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94acfba-1f89-4509-9acf-4977b3f100f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x\" (UID: \"a94acfba-1f89-4509-9acf-4977b3f100f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:22.497334 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.497294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:22.641752 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.641725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"]
Jul 16 22:58:22.643737 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:58:22.643707    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94acfba_1f89_4509_9acf_4977b3f100f9.slice/crio-112d2deb8f007b4382c19997b913ef3367a814d52661ff46f937f38c1cf34369 WatchSource:0}: Error finding container 112d2deb8f007b4382c19997b913ef3367a814d52661ff46f937f38c1cf34369: Status 404 returned error can't find the container with id 112d2deb8f007b4382c19997b913ef3367a814d52661ff46f937f38c1cf34369
Jul 16 22:58:22.894405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.894379    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" event={"ID":"a94acfba-1f89-4509-9acf-4977b3f100f9","Type":"ContainerStarted","Data":"112d2deb8f007b4382c19997b913ef3367a814d52661ff46f937f38c1cf34369"}
Jul 16 22:58:22.896273 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:22.896254    2564 scope.go:117] "RemoveContainer" containerID="3a8965b1a3eede82dc72b65b3bc8562de7c1761fadbb27480297778243cd4535"
Jul 16 22:58:22.896479 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:22.896458    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-qvwxs_odh-ai-gateway-infra(0d877c6f-6c3f-4d32-9bbd-4860b07158be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be"
Jul 16 22:58:23.902345 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:23.902305    2564 generic.go:356] "Generic (PLEG): container finished" podID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerID="40b983ba14baa3cb944e0272f825af92d7fdc45b00f04925d6d82f9b01df6ec7" exitCode=6
Jul 16 22:58:23.902755 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:23.902396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" event={"ID":"a94acfba-1f89-4509-9acf-4977b3f100f9","Type":"ContainerDied","Data":"40b983ba14baa3cb944e0272f825af92d7fdc45b00f04925d6d82f9b01df6ec7"}
Jul 16 22:58:23.902755 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:23.902597    2564 scope.go:117] "RemoveContainer" containerID="40b983ba14baa3cb944e0272f825af92d7fdc45b00f04925d6d82f9b01df6ec7"
Jul 16 22:58:24.909345 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:24.909309    2564 generic.go:356] "Generic (PLEG): container finished" podID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerID="666f17babd5773a00d82b8c8ec6d9f15945e8abf1eb87ca10dd70f3ea6d73e12" exitCode=6
Jul 16 22:58:24.909756 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:24.909351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" event={"ID":"a94acfba-1f89-4509-9acf-4977b3f100f9","Type":"ContainerDied","Data":"666f17babd5773a00d82b8c8ec6d9f15945e8abf1eb87ca10dd70f3ea6d73e12"}
Jul 16 22:58:24.909756 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:24.909407    2564 scope.go:117] "RemoveContainer" containerID="40b983ba14baa3cb944e0272f825af92d7fdc45b00f04925d6d82f9b01df6ec7"
Jul 16 22:58:24.909756 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:24.909619    2564 scope.go:117] "RemoveContainer" containerID="666f17babd5773a00d82b8c8ec6d9f15945e8abf1eb87ca10dd70f3ea6d73e12"
Jul 16 22:58:24.909942 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:24.909814    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x_odh-ai-gateway-infra(a94acfba-1f89-4509-9acf-4977b3f100f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9"
Jul 16 22:58:25.921700 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:25.921670    2564 scope.go:117] "RemoveContainer" containerID="666f17babd5773a00d82b8c8ec6d9f15945e8abf1eb87ca10dd70f3ea6d73e12"
Jul 16 22:58:25.922055 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:25.921866    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x_odh-ai-gateway-infra(a94acfba-1f89-4509-9acf-4977b3f100f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9"
Jul 16 22:58:26.856071 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:26.856033    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"]
Jul 16 22:58:26.860591 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:26.860573    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:26.867691 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:26.867542    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"]
Jul 16 22:58:26.938196 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:26.938161    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ae3972d-d15f-4f30-9c28-59010c279e5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz\" (UID: \"4ae3972d-d15f-4f30-9c28-59010c279e5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:27.038633 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.038595    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ae3972d-d15f-4f30-9c28-59010c279e5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz\" (UID: \"4ae3972d-d15f-4f30-9c28-59010c279e5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:27.039284 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.039260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ae3972d-d15f-4f30-9c28-59010c279e5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz\" (UID: \"4ae3972d-d15f-4f30-9c28-59010c279e5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:27.171900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.171806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:27.302494 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.302467    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"]
Jul 16 22:58:27.304524 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:58:27.304488    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae3972d_d15f_4f30_9c28_59010c279e5c.slice/crio-e50a70a9603f2d51356065fd2c32bfb9ecbcd23acc29deb928697b315276c1df WatchSource:0}: Error finding container e50a70a9603f2d51356065fd2c32bfb9ecbcd23acc29deb928697b315276c1df: Status 404 returned error can't find the container with id e50a70a9603f2d51356065fd2c32bfb9ecbcd23acc29deb928697b315276c1df
Jul 16 22:58:27.596209 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.596171    2564 scope.go:117] "RemoveContainer" containerID="1b6a4f6b1f6567ae1c19c3d7a5792dbe402e32b17b6c519c3113bf31983362df"
Jul 16 22:58:27.932858 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.932823    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerID="f795cf882167debc5a89d2e590819c38eb7ea6b75b5b4a456ef8570af85558a2" exitCode=6
Jul 16 22:58:27.933007 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.932908    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" event={"ID":"4ae3972d-d15f-4f30-9c28-59010c279e5c","Type":"ContainerDied","Data":"f795cf882167debc5a89d2e590819c38eb7ea6b75b5b4a456ef8570af85558a2"}
Jul 16 22:58:27.933007 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.932960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" event={"ID":"4ae3972d-d15f-4f30-9c28-59010c279e5c","Type":"ContainerStarted","Data":"e50a70a9603f2d51356065fd2c32bfb9ecbcd23acc29deb928697b315276c1df"}
Jul 16 22:58:27.933127 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.933045    2564 scope.go:117] "RemoveContainer" containerID="f795cf882167debc5a89d2e590819c38eb7ea6b75b5b4a456ef8570af85558a2"
Jul 16 22:58:27.934807 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:27.934776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerStarted","Data":"4d2207ce30021ba4376a3442b3d6041d115c6cdf8543d7fcb2932458b72f7cc9"}
Jul 16 22:58:28.942422 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.942329    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerID="05d6faa9b7dd51f7e195ac17f3f0d0dddd01eb5dc10430575e9da4e06f01ef8b" exitCode=6
Jul 16 22:58:28.942886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.942415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" event={"ID":"4ae3972d-d15f-4f30-9c28-59010c279e5c","Type":"ContainerDied","Data":"05d6faa9b7dd51f7e195ac17f3f0d0dddd01eb5dc10430575e9da4e06f01ef8b"}
Jul 16 22:58:28.942886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.942475    2564 scope.go:117] "RemoveContainer" containerID="f795cf882167debc5a89d2e590819c38eb7ea6b75b5b4a456ef8570af85558a2"
Jul 16 22:58:28.942886 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.942700    2564 scope.go:117] "RemoveContainer" containerID="05d6faa9b7dd51f7e195ac17f3f0d0dddd01eb5dc10430575e9da4e06f01ef8b"
Jul 16 22:58:28.943028 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:28.942924    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz_odh-ai-gateway-infra(4ae3972d-d15f-4f30-9c28-59010c279e5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c"
Jul 16 22:58:28.944326 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.944304    2564 generic.go:356] "Generic (PLEG): container finished" podID="af33c376-b77e-43d9-8d0c-70067acc237d" containerID="4d2207ce30021ba4376a3442b3d6041d115c6cdf8543d7fcb2932458b72f7cc9" exitCode=6
Jul 16 22:58:28.944449 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.944383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerDied","Data":"4d2207ce30021ba4376a3442b3d6041d115c6cdf8543d7fcb2932458b72f7cc9"}
Jul 16 22:58:28.944669 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.944655    2564 scope.go:117] "RemoveContainer" containerID="4d2207ce30021ba4376a3442b3d6041d115c6cdf8543d7fcb2932458b72f7cc9"
Jul 16 22:58:28.944826 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:28.944809    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l_odh-ai-gateway-infra(af33c376-b77e-43d9-8d0c-70067acc237d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" podUID="af33c376-b77e-43d9-8d0c-70067acc237d"
Jul 16 22:58:28.953547 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.953531    2564 scope.go:117] "RemoveContainer" containerID="1b6a4f6b1f6567ae1c19c3d7a5792dbe402e32b17b6c519c3113bf31983362df"
Jul 16 22:58:28.986635 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:28.986604    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"]
Jul 16 22:58:29.951636 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:29.951606    2564 scope.go:117] "RemoveContainer" containerID="05d6faa9b7dd51f7e195ac17f3f0d0dddd01eb5dc10430575e9da4e06f01ef8b"
Jul 16 22:58:29.952065 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:29.951854    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz_odh-ai-gateway-infra(4ae3972d-d15f-4f30-9c28-59010c279e5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c"
Jul 16 22:58:30.082364 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.082339    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:30.167234 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.167199    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af33c376-b77e-43d9-8d0c-70067acc237d-maas-api-service-ca\") pod \"af33c376-b77e-43d9-8d0c-70067acc237d\" (UID: \"af33c376-b77e-43d9-8d0c-70067acc237d\") "
Jul 16 22:58:30.167563 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.167540    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af33c376-b77e-43d9-8d0c-70067acc237d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "af33c376-b77e-43d9-8d0c-70067acc237d" (UID: "af33c376-b77e-43d9-8d0c-70067acc237d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:58:30.268829 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.268741    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af33c376-b77e-43d9-8d0c-70067acc237d-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:58:30.959498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.959463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l" event={"ID":"af33c376-b77e-43d9-8d0c-70067acc237d","Type":"ContainerDied","Data":"8039ed01dc6a7fc1e731fe613b5adba618b1731f3843fb1a5149441286848690"}
Jul 16 22:58:30.959498 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.959497    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"
Jul 16 22:58:30.960023 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.959526    2564 scope.go:117] "RemoveContainer" containerID="4d2207ce30021ba4376a3442b3d6041d115c6cdf8543d7fcb2932458b72f7cc9"
Jul 16 22:58:30.983634 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.983601    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"]
Jul 16 22:58:30.987889 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:30.987859    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nhk2l"]
Jul 16 22:58:31.601630 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:31.601596    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" path="/var/lib/kubelet/pods/af33c376-b77e-43d9-8d0c-70067acc237d/volumes"
Jul 16 22:58:34.596414 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:34.596385    2564 scope.go:117] "RemoveContainer" containerID="3a8965b1a3eede82dc72b65b3bc8562de7c1761fadbb27480297778243cd4535"
Jul 16 22:58:34.982983 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:34.982947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerStarted","Data":"2ab4a756383987a13b0871968eaaa6dcd5f65cc453e191bd94c244739d8fdbef"}
Jul 16 22:58:35.990065 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:35.990033    2564 generic.go:356] "Generic (PLEG): container finished" podID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerID="2ab4a756383987a13b0871968eaaa6dcd5f65cc453e191bd94c244739d8fdbef" exitCode=6
Jul 16 22:58:35.990065 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:35.990069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerDied","Data":"2ab4a756383987a13b0871968eaaa6dcd5f65cc453e191bd94c244739d8fdbef"}
Jul 16 22:58:35.990600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:35.990116    2564 scope.go:117] "RemoveContainer" containerID="3a8965b1a3eede82dc72b65b3bc8562de7c1761fadbb27480297778243cd4535"
Jul 16 22:58:35.990600 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:35.990396    2564 scope.go:117] "RemoveContainer" containerID="2ab4a756383987a13b0871968eaaa6dcd5f65cc453e191bd94c244739d8fdbef"
Jul 16 22:58:35.990677 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:35.990601    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-qvwxs_odh-ai-gateway-infra(0d877c6f-6c3f-4d32-9bbd-4860b07158be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be"
Jul 16 22:58:36.020057 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:36.020029    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"]
Jul 16 22:58:37.124443 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:37.124419    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:37.230078 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:37.230040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d877c6f-6c3f-4d32-9bbd-4860b07158be-maas-api-service-ca\") pod \"0d877c6f-6c3f-4d32-9bbd-4860b07158be\" (UID: \"0d877c6f-6c3f-4d32-9bbd-4860b07158be\") "
Jul 16 22:58:37.230436 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:37.230403    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d877c6f-6c3f-4d32-9bbd-4860b07158be-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d877c6f-6c3f-4d32-9bbd-4860b07158be" (UID: "0d877c6f-6c3f-4d32-9bbd-4860b07158be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:58:37.331788 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:37.331718    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d877c6f-6c3f-4d32-9bbd-4860b07158be-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:58:38.002934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:38.002901    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs" event={"ID":"0d877c6f-6c3f-4d32-9bbd-4860b07158be","Type":"ContainerDied","Data":"58e323ae6ec23e36f7e29cf7e132b2e2cc1d8ac43f93fb0d7764432e62b8f1bb"}
Jul 16 22:58:38.002934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:38.002933    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"
Jul 16 22:58:38.003174 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:38.002963    2564 scope.go:117] "RemoveContainer" containerID="2ab4a756383987a13b0871968eaaa6dcd5f65cc453e191bd94c244739d8fdbef"
Jul 16 22:58:38.023566 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:38.023536    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"]
Jul 16 22:58:38.029151 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:38.029115    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-qvwxs"]
Jul 16 22:58:39.602501 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:39.602465    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" path="/var/lib/kubelet/pods/0d877c6f-6c3f-4d32-9bbd-4860b07158be/volumes"
Jul 16 22:58:40.596386 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:40.596341    2564 scope.go:117] "RemoveContainer" containerID="666f17babd5773a00d82b8c8ec6d9f15945e8abf1eb87ca10dd70f3ea6d73e12"
Jul 16 22:58:41.021915 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:41.021879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" event={"ID":"a94acfba-1f89-4509-9acf-4977b3f100f9","Type":"ContainerStarted","Data":"9259afd65cde162ae59e3419bbed023d27b01ed9c833010ee0ff8a9b5954c7ca"}
Jul 16 22:58:41.041001 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:41.040940    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" podStartSLOduration=19.040924484 podStartE2EDuration="19.040924484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:58:41.038360034 +0000 UTC m=+2477.358916106" watchObservedRunningTime="2026-07-16 22:58:41.040924484 +0000 UTC m=+2477.361480555"
Jul 16 22:58:42.028751 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:42.028717    2564 generic.go:356] "Generic (PLEG): container finished" podID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerID="9259afd65cde162ae59e3419bbed023d27b01ed9c833010ee0ff8a9b5954c7ca" exitCode=6
Jul 16 22:58:42.029272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:42.028789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" event={"ID":"a94acfba-1f89-4509-9acf-4977b3f100f9","Type":"ContainerDied","Data":"9259afd65cde162ae59e3419bbed023d27b01ed9c833010ee0ff8a9b5954c7ca"}
Jul 16 22:58:42.029272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:42.028848    2564 scope.go:117] "RemoveContainer" containerID="666f17babd5773a00d82b8c8ec6d9f15945e8abf1eb87ca10dd70f3ea6d73e12"
Jul 16 22:58:42.029272 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:42.029113    2564 scope.go:117] "RemoveContainer" containerID="9259afd65cde162ae59e3419bbed023d27b01ed9c833010ee0ff8a9b5954c7ca"
Jul 16 22:58:42.029412 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:42.029348    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x_odh-ai-gateway-infra(a94acfba-1f89-4509-9acf-4977b3f100f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9"
Jul 16 22:58:42.059743 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:42.059700    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"]
Jul 16 22:58:43.170017 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:43.169993    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:43.283397 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:43.283357    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94acfba-1f89-4509-9acf-4977b3f100f9-maas-api-service-ca\") pod \"a94acfba-1f89-4509-9acf-4977b3f100f9\" (UID: \"a94acfba-1f89-4509-9acf-4977b3f100f9\") "
Jul 16 22:58:43.283729 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:43.283705    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94acfba-1f89-4509-9acf-4977b3f100f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a94acfba-1f89-4509-9acf-4977b3f100f9" (UID: "a94acfba-1f89-4509-9acf-4977b3f100f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:58:43.384637 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:43.384552    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94acfba-1f89-4509-9acf-4977b3f100f9-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:58:43.386514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:43.386491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:58:43.602502 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:43.602463    2564 scope.go:117] "RemoveContainer" containerID="05d6faa9b7dd51f7e195ac17f3f0d0dddd01eb5dc10430575e9da4e06f01ef8b"
Jul 16 22:58:44.045639 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.045606    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"
Jul 16 22:58:44.045838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.045635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x" event={"ID":"a94acfba-1f89-4509-9acf-4977b3f100f9","Type":"ContainerDied","Data":"112d2deb8f007b4382c19997b913ef3367a814d52661ff46f937f38c1cf34369"}
Jul 16 22:58:44.045838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.045711    2564 scope.go:117] "RemoveContainer" containerID="9259afd65cde162ae59e3419bbed023d27b01ed9c833010ee0ff8a9b5954c7ca"
Jul 16 22:58:44.047810 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.047744    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" event={"ID":"4ae3972d-d15f-4f30-9c28-59010c279e5c","Type":"ContainerStarted","Data":"3cf16633dad7cd15dc67144a33a6b913b4b87d8c067d19c76595727f6e82e09c"}
Jul 16 22:58:44.080564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.080519    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"]
Jul 16 22:58:44.091973 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.091932    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-bhv5x"]
Jul 16 22:58:44.123163 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:44.123083    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" podStartSLOduration=18.123063225 podStartE2EDuration="18.123063225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:58:44.120076444 +0000 UTC m=+2480.440632517" watchObservedRunningTime="2026-07-16 22:58:44.123063225 +0000 UTC m=+2480.443619297"
Jul 16 22:58:45.055709 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:45.055673    2564 generic.go:356] "Generic (PLEG): container finished" podID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerID="3cf16633dad7cd15dc67144a33a6b913b4b87d8c067d19c76595727f6e82e09c" exitCode=6
Jul 16 22:58:45.056214 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:45.055753    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" event={"ID":"4ae3972d-d15f-4f30-9c28-59010c279e5c","Type":"ContainerDied","Data":"3cf16633dad7cd15dc67144a33a6b913b4b87d8c067d19c76595727f6e82e09c"}
Jul 16 22:58:45.056214 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:45.055827    2564 scope.go:117] "RemoveContainer" containerID="05d6faa9b7dd51f7e195ac17f3f0d0dddd01eb5dc10430575e9da4e06f01ef8b"
Jul 16 22:58:45.056214 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:45.056200    2564 scope.go:117] "RemoveContainer" containerID="3cf16633dad7cd15dc67144a33a6b913b4b87d8c067d19c76595727f6e82e09c"
Jul 16 22:58:45.056434 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:58:45.056412    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz_odh-ai-gateway-infra(4ae3972d-d15f-4f30-9c28-59010c279e5c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c"
Jul 16 22:58:45.132316 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:45.132275    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"]
Jul 16 22:58:45.601942 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:45.601899    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" path="/var/lib/kubelet/pods/a94acfba-1f89-4509-9acf-4977b3f100f9/volumes"
Jul 16 22:58:46.214934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:46.214908    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:46.314846 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:46.314748    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ae3972d-d15f-4f30-9c28-59010c279e5c-maas-api-service-ca\") pod \"4ae3972d-d15f-4f30-9c28-59010c279e5c\" (UID: \"4ae3972d-d15f-4f30-9c28-59010c279e5c\") "
Jul 16 22:58:46.315195 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:46.315169    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ae3972d-d15f-4f30-9c28-59010c279e5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ae3972d-d15f-4f30-9c28-59010c279e5c" (UID: "4ae3972d-d15f-4f30-9c28-59010c279e5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:58:46.415362 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:46.415326    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ae3972d-d15f-4f30-9c28-59010c279e5c-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:58:47.068516 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:47.068482    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz" event={"ID":"4ae3972d-d15f-4f30-9c28-59010c279e5c","Type":"ContainerDied","Data":"e50a70a9603f2d51356065fd2c32bfb9ecbcd23acc29deb928697b315276c1df"}
Jul 16 22:58:47.068516 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:47.068514    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"
Jul 16 22:58:47.068747 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:47.068543    2564 scope.go:117] "RemoveContainer" containerID="3cf16633dad7cd15dc67144a33a6b913b4b87d8c067d19c76595727f6e82e09c"
Jul 16 22:58:47.089877 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:47.089850    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"]
Jul 16 22:58:47.099084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:47.096007    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-s72dz"]
Jul 16 22:58:47.602016 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:47.601982    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" path="/var/lib/kubelet/pods/4ae3972d-d15f-4f30-9c28-59010c279e5c/volumes"
Jul 16 22:58:52.885927 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:52.885898    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:58:55.931187 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931151    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"]
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931488    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931500    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931514    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931519    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931526    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931531    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931538    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931542    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931556    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931563    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931574    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931579    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931585    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.931589 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931589    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931602    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931607    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931613    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931618    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931623    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931628    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931686    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931694    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931701    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931708    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931717    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931725    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931731    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931737    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:58:55.932116 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.931742    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:58:55.935936 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.935917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:55.938888 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.938865    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bd998308-openshift-default-dockercfg-c957r\""
Jul 16 22:58:55.947453 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:55.947429    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"]
Jul 16 22:58:56.092792 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184855ae-c7f2-40b8-af71-5df074137269-istio-podinfo\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.092996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-istio-token\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.092996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-data\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.092996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184855ae-c7f2-40b8-af71-5df074137269-istiod-ca-cert\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.092996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcs4p\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-kube-api-access-mcs4p\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.092996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-certs\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.092996 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.092993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-envoy\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.093282 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.093021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-credential-socket\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.093282 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.093070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-socket\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184855ae-c7f2-40b8-af71-5df074137269-istio-podinfo\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-istio-token\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194153    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-data\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184855ae-c7f2-40b8-af71-5df074137269-istiod-ca-cert\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcs4p\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-kube-api-access-mcs4p\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-certs\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194596 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-envoy\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194596 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-credential-socket\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194596 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-data\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194596 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-socket\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194827 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194760    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-certs\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194827 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-credential-socket\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.194945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.194830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-socket\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.195081 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.195058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184855ae-c7f2-40b8-af71-5df074137269-istiod-ca-cert\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.196568 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.196548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-envoy\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.196680 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.196627    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184855ae-c7f2-40b8-af71-5df074137269-istio-podinfo\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.207327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.207304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcs4p\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-kube-api-access-mcs4p\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.207327 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.207315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-istio-token\") pod \"e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") " pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.246318 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.246287    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:56.386716 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.386690    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"]
Jul 16 22:58:56.387319 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:58:56.387294    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184855ae_c7f2_40b8_af71_5df074137269.slice/crio-338e7c5ac47cd30481b9bcfa94218ffc5dea86322c6684281f0c89bf105a1a4a WatchSource:0}: Error finding container 338e7c5ac47cd30481b9bcfa94218ffc5dea86322c6684281f0c89bf105a1a4a: Status 404 returned error can't find the container with id 338e7c5ac47cd30481b9bcfa94218ffc5dea86322c6684281f0c89bf105a1a4a
Jul 16 22:58:56.389426 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.389392    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:58:56.389523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.389459    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:58:56.389523 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:56.389488    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:58:57.122328 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:57.122294    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" event={"ID":"184855ae-c7f2-40b8-af71-5df074137269","Type":"ContainerStarted","Data":"6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"}
Jul 16 22:58:57.122729 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:57.122348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" event={"ID":"184855ae-c7f2-40b8-af71-5df074137269","Type":"ContainerStarted","Data":"338e7c5ac47cd30481b9bcfa94218ffc5dea86322c6684281f0c89bf105a1a4a"}
Jul 16 22:58:57.147571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:57.147520    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" podStartSLOduration=2.147503187 podStartE2EDuration="2.147503187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:58:57.144861471 +0000 UTC m=+2493.465417545" watchObservedRunningTime="2026-07-16 22:58:57.147503187 +0000 UTC m=+2493.468059263"
Jul 16 22:58:57.246753 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:57.246717    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:57.251626 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:57.251596    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:58.127462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:58.127430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:58:58.128468 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:58:58.128445    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:03.698103 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:03.698063    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:59:08.754676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.754640    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"]
Jul 16 22:59:08.755080 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755003    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:59:08.755080 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755014    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:59:08.755080 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755033    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:59:08.755080 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755039    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d877c6f-6c3f-4d32-9bbd-4860b07158be" containerName="revoke-keys"
Jul 16 22:59:08.755253 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755087    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af33c376-b77e-43d9-8d0c-70067acc237d" containerName="revoke-keys"
Jul 16 22:59:08.755253 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755097    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94acfba-1f89-4509-9acf-4977b3f100f9" containerName="revoke-keys"
Jul 16 22:59:08.755253 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.755104    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae3972d-d15f-4f30-9c28-59010c279e5c" containerName="revoke-keys"
Jul 16 22:59:08.758124 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.758108    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:08.760659 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.760635    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:59:08.760809 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.760637    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:59:08.766500 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.766028    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"]
Jul 16 22:59:08.915050 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:08.914994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7000c12b-0a24-4d0b-8147-12334bb4f30e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-92hkw\" (UID: \"7000c12b-0a24-4d0b-8147-12334bb4f30e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:09.016468 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:09.016376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7000c12b-0a24-4d0b-8147-12334bb4f30e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-92hkw\" (UID: \"7000c12b-0a24-4d0b-8147-12334bb4f30e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:09.016961 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:09.016941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7000c12b-0a24-4d0b-8147-12334bb4f30e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-92hkw\" (UID: \"7000c12b-0a24-4d0b-8147-12334bb4f30e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:09.070897 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:09.070849    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:09.219901 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:09.219879    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"]
Jul 16 22:59:09.222834 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:09.222806    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7000c12b_0a24_4d0b_8147_12334bb4f30e.slice/crio-805eeb878ba33387b7bdc0f76e7df15457704e4cd664e0d2ea85359db1d9adc2 WatchSource:0}: Error finding container 805eeb878ba33387b7bdc0f76e7df15457704e4cd664e0d2ea85359db1d9adc2: Status 404 returned error can't find the container with id 805eeb878ba33387b7bdc0f76e7df15457704e4cd664e0d2ea85359db1d9adc2
Jul 16 22:59:10.198193 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.198163    2564 generic.go:356] "Generic (PLEG): container finished" podID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerID="21b9a658c12f081ab155af472ea7ff863b3b48795d30371156f247652aa49b38" exitCode=6
Jul 16 22:59:10.198592 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.198243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" event={"ID":"7000c12b-0a24-4d0b-8147-12334bb4f30e","Type":"ContainerDied","Data":"21b9a658c12f081ab155af472ea7ff863b3b48795d30371156f247652aa49b38"}
Jul 16 22:59:10.198592 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.198289    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" event={"ID":"7000c12b-0a24-4d0b-8147-12334bb4f30e","Type":"ContainerStarted","Data":"805eeb878ba33387b7bdc0f76e7df15457704e4cd664e0d2ea85359db1d9adc2"}
Jul 16 22:59:10.198592 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.198452    2564 scope.go:117] "RemoveContainer" containerID="21b9a658c12f081ab155af472ea7ff863b3b48795d30371156f247652aa49b38"
Jul 16 22:59:10.236968 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.236935    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"]
Jul 16 22:59:10.241932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.241903    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:10.249302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.248726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"]
Jul 16 22:59:10.328059 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.328016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c640b184-4f44-4f9a-aaac-e04735e525c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz\" (UID: \"c640b184-4f44-4f9a-aaac-e04735e525c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:10.429462 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.429427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c640b184-4f44-4f9a-aaac-e04735e525c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz\" (UID: \"c640b184-4f44-4f9a-aaac-e04735e525c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:10.430007 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.429989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c640b184-4f44-4f9a-aaac-e04735e525c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz\" (UID: \"c640b184-4f44-4f9a-aaac-e04735e525c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:10.559258 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.559175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:10.701957 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:10.701926    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc640b184_4f44_4f9a_aaac_e04735e525c8.slice/crio-a988f8dd0418ae0837e2867cff602e0bf02b4cbca49455534e95585de4439b7a WatchSource:0}: Error finding container a988f8dd0418ae0837e2867cff602e0bf02b4cbca49455534e95585de4439b7a: Status 404 returned error can't find the container with id a988f8dd0418ae0837e2867cff602e0bf02b4cbca49455534e95585de4439b7a
Jul 16 22:59:10.702263 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:10.702241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"]
Jul 16 22:59:11.012889 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.012844    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"]
Jul 16 22:59:11.016347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.016322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:11.035185 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.035148    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"]
Jul 16 22:59:11.136656 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.136618    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5395fdc1-814a-44a5-a85c-a3af4d3fa1bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bd998308-jm4tn\" (UID: \"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:11.205769 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.205727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" event={"ID":"c640b184-4f44-4f9a-aaac-e04735e525c8","Type":"ContainerStarted","Data":"564f8b0b344e868d9098d89168bc84a9c0eba354f5d2557d3de43d658b2e28f8"}
Jul 16 22:59:11.206266 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.205793    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" event={"ID":"c640b184-4f44-4f9a-aaac-e04735e525c8","Type":"ContainerStarted","Data":"a988f8dd0418ae0837e2867cff602e0bf02b4cbca49455534e95585de4439b7a"}
Jul 16 22:59:11.207717 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.207693    2564 generic.go:356] "Generic (PLEG): container finished" podID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerID="d16824ede5ccdc9a66e4f068749361ead69c8002b5bb9497ae00a91533c7c986" exitCode=6
Jul 16 22:59:11.207833 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.207751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" event={"ID":"7000c12b-0a24-4d0b-8147-12334bb4f30e","Type":"ContainerDied","Data":"d16824ede5ccdc9a66e4f068749361ead69c8002b5bb9497ae00a91533c7c986"}
Jul 16 22:59:11.207833 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.207797    2564 scope.go:117] "RemoveContainer" containerID="21b9a658c12f081ab155af472ea7ff863b3b48795d30371156f247652aa49b38"
Jul 16 22:59:11.207979 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.207961    2564 scope.go:117] "RemoveContainer" containerID="d16824ede5ccdc9a66e4f068749361ead69c8002b5bb9497ae00a91533c7c986"
Jul 16 22:59:11.208213 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:11.208194    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-92hkw_odh-ai-gateway-infra(7000c12b-0a24-4d0b-8147-12334bb4f30e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e"
Jul 16 22:59:11.226344 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.226298    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" podStartSLOduration=1.226276092 podStartE2EDuration="1.226276092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:59:11.223540704 +0000 UTC m=+2507.544096779" watchObservedRunningTime="2026-07-16 22:59:11.226276092 +0000 UTC m=+2507.546832231"
Jul 16 22:59:11.238194 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.238156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5395fdc1-814a-44a5-a85c-a3af4d3fa1bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bd998308-jm4tn\" (UID: \"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:11.238762 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.238741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5395fdc1-814a-44a5-a85c-a3af4d3fa1bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bd998308-jm4tn\" (UID: \"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:11.326682 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.326649    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:11.465105 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:11.465080    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"]
Jul 16 22:59:11.466656 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:11.466624    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5395fdc1_814a_44a5_a85c_a3af4d3fa1bf.slice/crio-dbf20cc7d1fca3f8667aeeafc9dc1b57df1e94ab0964c6131210db33a8e38399 WatchSource:0}: Error finding container dbf20cc7d1fca3f8667aeeafc9dc1b57df1e94ab0964c6131210db33a8e38399: Status 404 returned error can't find the container with id dbf20cc7d1fca3f8667aeeafc9dc1b57df1e94ab0964c6131210db33a8e38399
Jul 16 22:59:12.215150 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.215120    2564 scope.go:117] "RemoveContainer" containerID="d16824ede5ccdc9a66e4f068749361ead69c8002b5bb9497ae00a91533c7c986"
Jul 16 22:59:12.215644 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:12.215404    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-92hkw_odh-ai-gateway-infra(7000c12b-0a24-4d0b-8147-12334bb4f30e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e"
Jul 16 22:59:12.216540 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.216518    2564 generic.go:356] "Generic (PLEG): container finished" podID="184855ae-c7f2-40b8-af71-5df074137269" containerID="6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7" exitCode=0
Jul 16 22:59:12.216625 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.216590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" event={"ID":"184855ae-c7f2-40b8-af71-5df074137269","Type":"ContainerDied","Data":"6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"}
Jul 16 22:59:12.216957 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.216937    2564 scope.go:117] "RemoveContainer" containerID="6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"
Jul 16 22:59:12.217961 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.217932    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:59:12.218051 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.218011    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:59:12.218099 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.218061    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:59:12.218179 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.218152    2564 generic.go:356] "Generic (PLEG): container finished" podID="5395fdc1-814a-44a5-a85c-a3af4d3fa1bf" containerID="f4a6f65e3dfbf7c5177f802954b0ee86cb484f6bb24e5953e7db6e45fbc8ac17" exitCode=0
Jul 16 22:59:12.218245 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.218215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn" event={"ID":"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf","Type":"ContainerDied","Data":"f4a6f65e3dfbf7c5177f802954b0ee86cb484f6bb24e5953e7db6e45fbc8ac17"}
Jul 16 22:59:12.218290 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.218255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn" event={"ID":"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf","Type":"ContainerStarted","Data":"dbf20cc7d1fca3f8667aeeafc9dc1b57df1e94ab0964c6131210db33a8e38399"}
Jul 16 22:59:12.219816 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.219796    2564 generic.go:356] "Generic (PLEG): container finished" podID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerID="564f8b0b344e868d9098d89168bc84a9c0eba354f5d2557d3de43d658b2e28f8" exitCode=6
Jul 16 22:59:12.219896 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.219851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" event={"ID":"c640b184-4f44-4f9a-aaac-e04735e525c8","Type":"ContainerDied","Data":"564f8b0b344e868d9098d89168bc84a9c0eba354f5d2557d3de43d658b2e28f8"}
Jul 16 22:59:12.220012 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:12.220000    2564 scope.go:117] "RemoveContainer" containerID="564f8b0b344e868d9098d89168bc84a9c0eba354f5d2557d3de43d658b2e28f8"
Jul 16 22:59:13.127699 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.127655    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:13.234270 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.234229    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" event={"ID":"184855ae-c7f2-40b8-af71-5df074137269","Type":"ContainerStarted","Data":"76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031"}
Jul 16 22:59:13.236355 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.236330    2564 generic.go:356] "Generic (PLEG): container finished" podID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerID="57848504710f28cc173dc3023cde2f0f8bd057ece3bd0cc2d61a5e19612247a7" exitCode=6
Jul 16 22:59:13.236488 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.236402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" event={"ID":"c640b184-4f44-4f9a-aaac-e04735e525c8","Type":"ContainerDied","Data":"57848504710f28cc173dc3023cde2f0f8bd057ece3bd0cc2d61a5e19612247a7"}
Jul 16 22:59:13.236488 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.236434    2564 scope.go:117] "RemoveContainer" containerID="564f8b0b344e868d9098d89168bc84a9c0eba354f5d2557d3de43d658b2e28f8"
Jul 16 22:59:13.236583 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.236526    2564 scope.go:117] "RemoveContainer" containerID="57848504710f28cc173dc3023cde2f0f8bd057ece3bd0cc2d61a5e19612247a7"
Jul 16 22:59:13.236717 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:13.236695    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz_odh-ai-gateway-infra(c640b184-4f44-4f9a-aaac-e04735e525c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8"
Jul 16 22:59:13.246430 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.246406    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:13.247779 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.247752    2564 patch_prober.go:28] interesting pod/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 16 22:59:13.247899 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.247800    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 16 22:59:13.415299 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.415277    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:13.560285 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.560187    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5395fdc1-814a-44a5-a85c-a3af4d3fa1bf-maas-api-service-ca\") pod \"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf\" (UID: \"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf\") "
Jul 16 22:59:13.560550 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.560526    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5395fdc1-814a-44a5-a85c-a3af4d3fa1bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5395fdc1-814a-44a5-a85c-a3af4d3fa1bf" (UID: "5395fdc1-814a-44a5-a85c-a3af4d3fa1bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:13.661184 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:13.661119    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5395fdc1-814a-44a5-a85c-a3af4d3fa1bf-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:14.243444 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:14.243409    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn"
Jul 16 22:59:14.243933 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:14.243446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bd998308-jm4tn" event={"ID":"5395fdc1-814a-44a5-a85c-a3af4d3fa1bf","Type":"ContainerDied","Data":"dbf20cc7d1fca3f8667aeeafc9dc1b57df1e94ab0964c6131210db33a8e38399"}
Jul 16 22:59:14.243933 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:14.243487    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbf20cc7d1fca3f8667aeeafc9dc1b57df1e94ab0964c6131210db33a8e38399"
Jul 16 22:59:14.245177 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:14.245157    2564 scope.go:117] "RemoveContainer" containerID="57848504710f28cc173dc3023cde2f0f8bd057ece3bd0cc2d61a5e19612247a7"
Jul 16 22:59:14.245415 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:14.245374    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz_odh-ai-gateway-infra(c640b184-4f44-4f9a-aaac-e04735e525c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8"
Jul 16 22:59:14.247093 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:14.247070    2564 patch_prober.go:28] interesting pod/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 16 22:59:14.247188 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:14.247109    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 16 22:59:15.246698 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:15.246660    2564 patch_prober.go:28] interesting pod/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 16 22:59:15.247088 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:15.246721    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 16 22:59:16.247677 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:16.247596    2564 patch_prober.go:28] interesting pod/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Jul 16 22:59:16.247677 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:16.247652    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Jul 16 22:59:17.250870 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:17.250836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:17.251358 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:17.251202    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:17.251822 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:17.251803    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:22.219263 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.219220    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"]
Jul 16 22:59:22.219648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.219607    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5395fdc1-814a-44a5-a85c-a3af4d3fa1bf" containerName="revoke-keys"
Jul 16 22:59:22.219648 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.219622    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5395fdc1-814a-44a5-a85c-a3af4d3fa1bf" containerName="revoke-keys"
Jul 16 22:59:22.219724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.219680    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5395fdc1-814a-44a5-a85c-a3af4d3fa1bf" containerName="revoke-keys"
Jul 16 22:59:22.222029 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.222012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:22.235033 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.235004    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"]
Jul 16 22:59:22.238819 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.238770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94609380-bdab-4fc1-90db-ff09046c8b5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2\" (UID: \"94609380-bdab-4fc1-90db-ff09046c8b5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:22.339969 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.339928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94609380-bdab-4fc1-90db-ff09046c8b5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2\" (UID: \"94609380-bdab-4fc1-90db-ff09046c8b5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:22.340564 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.340546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94609380-bdab-4fc1-90db-ff09046c8b5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2\" (UID: \"94609380-bdab-4fc1-90db-ff09046c8b5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:22.532163 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.532028    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:22.666220 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:22.666186    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"]
Jul 16 22:59:22.667602 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:22.667573    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94609380_bdab_4fc1_90db_ff09046c8b5e.slice/crio-615ed51a008bdb1a5d109396f9d29a2197eb1847b5c01ae9624f81a240ddc174 WatchSource:0}: Error finding container 615ed51a008bdb1a5d109396f9d29a2197eb1847b5c01ae9624f81a240ddc174: Status 404 returned error can't find the container with id 615ed51a008bdb1a5d109396f9d29a2197eb1847b5c01ae9624f81a240ddc174
Jul 16 22:59:23.300681 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:23.300648    2564 generic.go:356] "Generic (PLEG): container finished" podID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerID="39fca3cc4a593e34d297256bd0cbc7be93a460bcb3df19ee0cc396a510b9c82e" exitCode=6
Jul 16 22:59:23.301019 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:23.300725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" event={"ID":"94609380-bdab-4fc1-90db-ff09046c8b5e","Type":"ContainerDied","Data":"39fca3cc4a593e34d297256bd0cbc7be93a460bcb3df19ee0cc396a510b9c82e"}
Jul 16 22:59:23.301019 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:23.300811    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" event={"ID":"94609380-bdab-4fc1-90db-ff09046c8b5e","Type":"ContainerStarted","Data":"615ed51a008bdb1a5d109396f9d29a2197eb1847b5c01ae9624f81a240ddc174"}
Jul 16 22:59:23.301019 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:23.300885    2564 scope.go:117] "RemoveContainer" containerID="39fca3cc4a593e34d297256bd0cbc7be93a460bcb3df19ee0cc396a510b9c82e"
Jul 16 22:59:23.596302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:23.596226    2564 scope.go:117] "RemoveContainer" containerID="d16824ede5ccdc9a66e4f068749361ead69c8002b5bb9497ae00a91533c7c986"
Jul 16 22:59:24.307806 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.307773    2564 generic.go:356] "Generic (PLEG): container finished" podID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerID="9365c8493922dcc69f049e219059269fd80d23f40da4348201ad3d7210cd9762" exitCode=6
Jul 16 22:59:24.308249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.307841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" event={"ID":"7000c12b-0a24-4d0b-8147-12334bb4f30e","Type":"ContainerDied","Data":"9365c8493922dcc69f049e219059269fd80d23f40da4348201ad3d7210cd9762"}
Jul 16 22:59:24.308249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.307903    2564 scope.go:117] "RemoveContainer" containerID="d16824ede5ccdc9a66e4f068749361ead69c8002b5bb9497ae00a91533c7c986"
Jul 16 22:59:24.308249 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.308217    2564 scope.go:117] "RemoveContainer" containerID="9365c8493922dcc69f049e219059269fd80d23f40da4348201ad3d7210cd9762"
Jul 16 22:59:24.308432 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:24.308407    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-92hkw_odh-ai-gateway-infra(7000c12b-0a24-4d0b-8147-12334bb4f30e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e"
Jul 16 22:59:24.309889 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.309867    2564 generic.go:356] "Generic (PLEG): container finished" podID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerID="319ec35765abe5ec2694f1db4267a0c3fc6f33c01b6a2a0cd70da4e31dc8559f" exitCode=6
Jul 16 22:59:24.310020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.309940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" event={"ID":"94609380-bdab-4fc1-90db-ff09046c8b5e","Type":"ContainerDied","Data":"319ec35765abe5ec2694f1db4267a0c3fc6f33c01b6a2a0cd70da4e31dc8559f"}
Jul 16 22:59:24.310111 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.310098    2564 scope.go:117] "RemoveContainer" containerID="319ec35765abe5ec2694f1db4267a0c3fc6f33c01b6a2a0cd70da4e31dc8559f"
Jul 16 22:59:24.310284 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:24.310267    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2_odh-ai-gateway-infra(94609380-bdab-4fc1-90db-ff09046c8b5e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e"
Jul 16 22:59:24.318624 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.318610    2564 scope.go:117] "RemoveContainer" containerID="39fca3cc4a593e34d297256bd0cbc7be93a460bcb3df19ee0cc396a510b9c82e"
Jul 16 22:59:24.585008 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:24.584974    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 22:59:25.318185 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:25.318161    2564 scope.go:117] "RemoveContainer" containerID="319ec35765abe5ec2694f1db4267a0c3fc6f33c01b6a2a0cd70da4e31dc8559f"
Jul 16 22:59:25.318553 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:25.318346    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2_odh-ai-gateway-infra(94609380-bdab-4fc1-90db-ff09046c8b5e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e"
Jul 16 22:59:25.341424 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:25.341395    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"]
Jul 16 22:59:25.477313 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:25.477284    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:25.570821 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:25.570725    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7000c12b-0a24-4d0b-8147-12334bb4f30e-maas-api-service-ca\") pod \"7000c12b-0a24-4d0b-8147-12334bb4f30e\" (UID: \"7000c12b-0a24-4d0b-8147-12334bb4f30e\") "
Jul 16 22:59:25.571104 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:25.571080    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7000c12b-0a24-4d0b-8147-12334bb4f30e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7000c12b-0a24-4d0b-8147-12334bb4f30e" (UID: "7000c12b-0a24-4d0b-8147-12334bb4f30e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:25.671742 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:25.671711    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7000c12b-0a24-4d0b-8147-12334bb4f30e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:26.324398 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.324372    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"
Jul 16 22:59:26.324820 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.324370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw" event={"ID":"7000c12b-0a24-4d0b-8147-12334bb4f30e","Type":"ContainerDied","Data":"805eeb878ba33387b7bdc0f76e7df15457704e4cd664e0d2ea85359db1d9adc2"}
Jul 16 22:59:26.324820 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.324507    2564 scope.go:117] "RemoveContainer" containerID="9365c8493922dcc69f049e219059269fd80d23f40da4348201ad3d7210cd9762"
Jul 16 22:59:26.341919 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.341895    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"]
Jul 16 22:59:26.350296 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.350266    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-92hkw"]
Jul 16 22:59:26.596405 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.596318    2564 scope.go:117] "RemoveContainer" containerID="57848504710f28cc173dc3023cde2f0f8bd057ece3bd0cc2d61a5e19612247a7"
Jul 16 22:59:26.906537 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906455    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"]
Jul 16 22:59:26.906804 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906782    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:26.906852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906806    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:26.906852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906816    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:26.906852 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906823    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:26.906945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906919    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:26.906945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.906929    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:26.908901 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.908884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:26.917202 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.917174    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"]
Jul 16 22:59:26.983280 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:26.983244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb4943-28d4-42ad-9589-4876b37c2117-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq\" (UID: \"d3cb4943-28d4-42ad-9589-4876b37c2117\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:27.084449 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.084419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb4943-28d4-42ad-9589-4876b37c2117-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq\" (UID: \"d3cb4943-28d4-42ad-9589-4876b37c2117\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:27.085028 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.085007    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb4943-28d4-42ad-9589-4876b37c2117-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq\" (UID: \"d3cb4943-28d4-42ad-9589-4876b37c2117\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:27.220168 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.220127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:27.336517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.336485    2564 generic.go:356] "Generic (PLEG): container finished" podID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerID="4e1594cfdc8199d43a29ccb4d9e81015084c72ad4d68e65414e6cb01615f491b" exitCode=6
Jul 16 22:59:27.336929 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.336542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" event={"ID":"c640b184-4f44-4f9a-aaac-e04735e525c8","Type":"ContainerDied","Data":"4e1594cfdc8199d43a29ccb4d9e81015084c72ad4d68e65414e6cb01615f491b"}
Jul 16 22:59:27.336929 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.336589    2564 scope.go:117] "RemoveContainer" containerID="57848504710f28cc173dc3023cde2f0f8bd057ece3bd0cc2d61a5e19612247a7"
Jul 16 22:59:27.336929 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.336876    2564 scope.go:117] "RemoveContainer" containerID="4e1594cfdc8199d43a29ccb4d9e81015084c72ad4d68e65414e6cb01615f491b"
Jul 16 22:59:27.337215 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:27.337176    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz_odh-ai-gateway-infra(c640b184-4f44-4f9a-aaac-e04735e525c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8"
Jul 16 22:59:27.356881 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.356828    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"]
Jul 16 22:59:27.359080 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:27.359051    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cb4943_28d4_42ad_9589_4876b37c2117.slice/crio-52d81fa215c510d4edc88807d8849efcb4e2e068935b7a438e09448303ba405b WatchSource:0}: Error finding container 52d81fa215c510d4edc88807d8849efcb4e2e068935b7a438e09448303ba405b: Status 404 returned error can't find the container with id 52d81fa215c510d4edc88807d8849efcb4e2e068935b7a438e09448303ba405b
Jul 16 22:59:27.602019 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:27.601988    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" path="/var/lib/kubelet/pods/7000c12b-0a24-4d0b-8147-12334bb4f30e/volumes"
Jul 16 22:59:28.343542 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.343459    2564 generic.go:356] "Generic (PLEG): container finished" podID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerID="c034289a3eb07c9bcc96227ed31e5884e0403335eedec85116d776e78d97638b" exitCode=6
Jul 16 22:59:28.343911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.343538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" event={"ID":"d3cb4943-28d4-42ad-9589-4876b37c2117","Type":"ContainerDied","Data":"c034289a3eb07c9bcc96227ed31e5884e0403335eedec85116d776e78d97638b"}
Jul 16 22:59:28.343911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.343580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" event={"ID":"d3cb4943-28d4-42ad-9589-4876b37c2117","Type":"ContainerStarted","Data":"52d81fa215c510d4edc88807d8849efcb4e2e068935b7a438e09448303ba405b"}
Jul 16 22:59:28.343911 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.343708    2564 scope.go:117] "RemoveContainer" containerID="c034289a3eb07c9bcc96227ed31e5884e0403335eedec85116d776e78d97638b"
Jul 16 22:59:28.368890 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.368865    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"]
Jul 16 22:59:28.498486 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.498456    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:28.597274 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.597208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c640b184-4f44-4f9a-aaac-e04735e525c8-maas-api-service-ca\") pod \"c640b184-4f44-4f9a-aaac-e04735e525c8\" (UID: \"c640b184-4f44-4f9a-aaac-e04735e525c8\") "
Jul 16 22:59:28.597495 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.597475    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c640b184-4f44-4f9a-aaac-e04735e525c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c640b184-4f44-4f9a-aaac-e04735e525c8" (UID: "c640b184-4f44-4f9a-aaac-e04735e525c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:28.698149 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:28.698106    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c640b184-4f44-4f9a-aaac-e04735e525c8-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:29.351724 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.351627    2564 generic.go:356] "Generic (PLEG): container finished" podID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerID="aba7dd0796568fa005e285dbb4b889a39d715bc1bfeccdec1df4c5a845e9740d" exitCode=6
Jul 16 22:59:29.352231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.351709    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" event={"ID":"d3cb4943-28d4-42ad-9589-4876b37c2117","Type":"ContainerDied","Data":"aba7dd0796568fa005e285dbb4b889a39d715bc1bfeccdec1df4c5a845e9740d"}
Jul 16 22:59:29.352231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.351784    2564 scope.go:117] "RemoveContainer" containerID="c034289a3eb07c9bcc96227ed31e5884e0403335eedec85116d776e78d97638b"
Jul 16 22:59:29.352231 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.351976    2564 scope.go:117] "RemoveContainer" containerID="aba7dd0796568fa005e285dbb4b889a39d715bc1bfeccdec1df4c5a845e9740d"
Jul 16 22:59:29.352390 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:29.352243    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq_odh-ai-gateway-infra(d3cb4943-28d4-42ad-9589-4876b37c2117)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117"
Jul 16 22:59:29.353043 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.353028    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"
Jul 16 22:59:29.353095 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.353064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz" event={"ID":"c640b184-4f44-4f9a-aaac-e04735e525c8","Type":"ContainerDied","Data":"a988f8dd0418ae0837e2867cff602e0bf02b4cbca49455534e95585de4439b7a"}
Jul 16 22:59:29.362479 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.362464    2564 scope.go:117] "RemoveContainer" containerID="4e1594cfdc8199d43a29ccb4d9e81015084c72ad4d68e65414e6cb01615f491b"
Jul 16 22:59:29.383934 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.383910    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"]
Jul 16 22:59:29.393110 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.393085    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-ptccz"]
Jul 16 22:59:29.602236 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:29.602157    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" path="/var/lib/kubelet/pods/c640b184-4f44-4f9a-aaac-e04735e525c8/volumes"
Jul 16 22:59:30.361013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:30.360973    2564 scope.go:117] "RemoveContainer" containerID="aba7dd0796568fa005e285dbb4b889a39d715bc1bfeccdec1df4c5a845e9740d"
Jul 16 22:59:30.361509 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:30.361225    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq_odh-ai-gateway-infra(d3cb4943-28d4-42ad-9589-4876b37c2117)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117"
Jul 16 22:59:36.596604 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:36.596567    2564 scope.go:117] "RemoveContainer" containerID="319ec35765abe5ec2694f1db4267a0c3fc6f33c01b6a2a0cd70da4e31dc8559f"
Jul 16 22:59:36.731507 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:36.731473    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"]
Jul 16 22:59:36.731764 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:36.731735    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy" containerID="cri-o://76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031" gracePeriod=30
Jul 16 22:59:37.401489 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:37.401453    2564 generic.go:356] "Generic (PLEG): container finished" podID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerID="c20d9cf1b0d2ba2fbfe5e9f4ce50d1a077e533d2568baf9e11364290640705ce" exitCode=6
Jul 16 22:59:37.401662 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:37.401532    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" event={"ID":"94609380-bdab-4fc1-90db-ff09046c8b5e","Type":"ContainerDied","Data":"c20d9cf1b0d2ba2fbfe5e9f4ce50d1a077e533d2568baf9e11364290640705ce"}
Jul 16 22:59:37.401662 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:37.401620    2564 scope.go:117] "RemoveContainer" containerID="319ec35765abe5ec2694f1db4267a0c3fc6f33c01b6a2a0cd70da4e31dc8559f"
Jul 16 22:59:37.401916 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:37.401899    2564 scope.go:117] "RemoveContainer" containerID="c20d9cf1b0d2ba2fbfe5e9f4ce50d1a077e533d2568baf9e11364290640705ce"
Jul 16 22:59:37.402189 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:37.402162    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2_odh-ai-gateway-infra(94609380-bdab-4fc1-90db-ff09046c8b5e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e"
Jul 16 22:59:38.448471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:38.448439    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"]
Jul 16 22:59:38.586215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:38.586186    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:38.690356 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:38.690325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94609380-bdab-4fc1-90db-ff09046c8b5e-maas-api-service-ca\") pod \"94609380-bdab-4fc1-90db-ff09046c8b5e\" (UID: \"94609380-bdab-4fc1-90db-ff09046c8b5e\") "
Jul 16 22:59:38.690696 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:38.690674    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94609380-bdab-4fc1-90db-ff09046c8b5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "94609380-bdab-4fc1-90db-ff09046c8b5e" (UID: "94609380-bdab-4fc1-90db-ff09046c8b5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:38.792024 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:38.791936    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94609380-bdab-4fc1-90db-ff09046c8b5e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:39.414705 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:39.414663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2" event={"ID":"94609380-bdab-4fc1-90db-ff09046c8b5e","Type":"ContainerDied","Data":"615ed51a008bdb1a5d109396f9d29a2197eb1847b5c01ae9624f81a240ddc174"}
Jul 16 22:59:39.414924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:39.414728    2564 scope.go:117] "RemoveContainer" containerID="c20d9cf1b0d2ba2fbfe5e9f4ce50d1a077e533d2568baf9e11364290640705ce"
Jul 16 22:59:39.414924 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:39.414688    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"
Jul 16 22:59:39.437082 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:39.437049    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"]
Jul 16 22:59:39.442232 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:39.442208    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sstc2"]
Jul 16 22:59:39.602439 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:39.602406    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" path="/var/lib/kubelet/pods/94609380-bdab-4fc1-90db-ff09046c8b5e/volumes"
Jul 16 22:59:40.597013 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:40.596985    2564 scope.go:117] "RemoveContainer" containerID="aba7dd0796568fa005e285dbb4b889a39d715bc1bfeccdec1df4c5a845e9740d"
Jul 16 22:59:41.427969 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:41.427933    2564 generic.go:356] "Generic (PLEG): container finished" podID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerID="80bccc215ff1058329e30cd533feab8d2702cbd586588361ea934b53429b877d" exitCode=6
Jul 16 22:59:41.428464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:41.428006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" event={"ID":"d3cb4943-28d4-42ad-9589-4876b37c2117","Type":"ContainerDied","Data":"80bccc215ff1058329e30cd533feab8d2702cbd586588361ea934b53429b877d"}
Jul 16 22:59:41.428464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:41.428069    2564 scope.go:117] "RemoveContainer" containerID="aba7dd0796568fa005e285dbb4b889a39d715bc1bfeccdec1df4c5a845e9740d"
Jul 16 22:59:41.428464 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:41.428431    2564 scope.go:117] "RemoveContainer" containerID="80bccc215ff1058329e30cd533feab8d2702cbd586588361ea934b53429b877d"
Jul 16 22:59:41.428678 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:41.428656    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq_odh-ai-gateway-infra(d3cb4943-28d4-42ad-9589-4876b37c2117)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117"
Jul 16 22:59:42.002313 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.002238    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:42.120586 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120542    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-data\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.120586 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184855ae-c7f2-40b8-af71-5df074137269-istiod-ca-cert\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.120838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120606    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-credential-socket\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.120838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-istio-token\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.120838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120696    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcs4p\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-kube-api-access-mcs4p\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.120838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120787    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-envoy\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.120838 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184855ae-c7f2-40b8-af71-5df074137269-istio-podinfo\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.121100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120844    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-certs\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.121100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120867    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-socket\") pod \"184855ae-c7f2-40b8-af71-5df074137269\" (UID: \"184855ae-c7f2-40b8-af71-5df074137269\") "
Jul 16 22:59:42.121100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.120973    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:42.121100 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.121032    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184855ae-c7f2-40b8-af71-5df074137269-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:42.121319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.121204    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184855ae-c7f2-40b8-af71-5df074137269-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.121319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.121222    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.121319 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.121235    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:42.121461 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.121318    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:42.121635 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.121614    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-data" (OuterVolumeSpecName: "istio-data") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:42.123023 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.122996    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/184855ae-c7f2-40b8-af71-5df074137269-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 22:59:42.123167 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.123145    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-kube-api-access-mcs4p" (OuterVolumeSpecName: "kube-api-access-mcs4p") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "kube-api-access-mcs4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:59:42.123361 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.123343    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-istio-token" (OuterVolumeSpecName: "istio-token") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:59:42.123454 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.123437    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "184855ae-c7f2-40b8-af71-5df074137269" (UID: "184855ae-c7f2-40b8-af71-5df074137269"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:42.222191 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222155    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.222191 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222193    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.222452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222209    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcs4p\" (UniqueName: \"kubernetes.io/projected/184855ae-c7f2-40b8-af71-5df074137269-kube-api-access-mcs4p\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.222452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222223    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.222452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222243    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184855ae-c7f2-40b8-af71-5df074137269-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.222452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222255    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.222452 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.222278    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184855ae-c7f2-40b8-af71-5df074137269-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.434870 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.434760    2564 generic.go:356] "Generic (PLEG): container finished" podID="184855ae-c7f2-40b8-af71-5df074137269" containerID="76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031" exitCode=0
Jul 16 22:59:42.434870 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.434843    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"
Jul 16 22:59:42.434870 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.434850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" event={"ID":"184855ae-c7f2-40b8-af71-5df074137269","Type":"ContainerDied","Data":"76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031"}
Jul 16 22:59:42.435514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.434939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x" event={"ID":"184855ae-c7f2-40b8-af71-5df074137269","Type":"ContainerDied","Data":"338e7c5ac47cd30481b9bcfa94218ffc5dea86322c6684281f0c89bf105a1a4a"}
Jul 16 22:59:42.435514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.434971    2564 scope.go:117] "RemoveContainer" containerID="76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031"
Jul 16 22:59:42.444447 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.444431    2564 scope.go:117] "RemoveContainer" containerID="6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"
Jul 16 22:59:42.452459 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.452440    2564 scope.go:117] "RemoveContainer" containerID="76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031"
Jul 16 22:59:42.452707 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:42.452689    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031\": container with ID starting with 76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031 not found: ID does not exist" containerID="76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031"
Jul 16 22:59:42.452763 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.452716    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031"} err="failed to get container status \"76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031\": rpc error: code = NotFound desc = could not find container \"76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031\": container with ID starting with 76c09306bee3939af3d226d2f918f3e62aec3981062b7527ecbd4b834a986031 not found: ID does not exist"
Jul 16 22:59:42.452763 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.452734    2564 scope.go:117] "RemoveContainer" containerID="6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"
Jul 16 22:59:42.452963 ip-10-0-136-7 kubenswrapper[2564]: E0716 22:59:42.452944    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7\": container with ID starting with 6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7 not found: ID does not exist" containerID="6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"
Jul 16 22:59:42.453005 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.452969    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7"} err="failed to get container status \"6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7\": rpc error: code = NotFound desc = could not find container \"6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7\": container with ID starting with 6e981dc930041849115bddc09ea0b5b0e3a1f847eca927116954d97e978c80c7 not found: ID does not exist"
Jul 16 22:59:42.456517 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.456495    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"]
Jul 16 22:59:42.460571 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.460545    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"]
Jul 16 22:59:42.467840 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.467819    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bd998308-openshift-default-dd65f478c-m5m8x"]
Jul 16 22:59:42.576945 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.576922    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:42.626020 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.625984    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb4943-28d4-42ad-9589-4876b37c2117-maas-api-service-ca\") pod \"d3cb4943-28d4-42ad-9589-4876b37c2117\" (UID: \"d3cb4943-28d4-42ad-9589-4876b37c2117\") "
Jul 16 22:59:42.626365 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.626341    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3cb4943-28d4-42ad-9589-4876b37c2117-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d3cb4943-28d4-42ad-9589-4876b37c2117" (UID: "d3cb4943-28d4-42ad-9589-4876b37c2117"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:42.727504 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:42.727465    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb4943-28d4-42ad-9589-4876b37c2117-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 22:59:43.443932 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.443901    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"
Jul 16 22:59:43.444419 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.443946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq" event={"ID":"d3cb4943-28d4-42ad-9589-4876b37c2117","Type":"ContainerDied","Data":"52d81fa215c510d4edc88807d8849efcb4e2e068935b7a438e09448303ba405b"}
Jul 16 22:59:43.444419 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.444009    2564 scope.go:117] "RemoveContainer" containerID="80bccc215ff1058329e30cd533feab8d2702cbd586588361ea934b53429b877d"
Jul 16 22:59:43.467645 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.467620    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"]
Jul 16 22:59:43.471944 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.471915    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-xr4xq"]
Jul 16 22:59:43.602411 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.602375    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="184855ae-c7f2-40b8-af71-5df074137269" path="/var/lib/kubelet/pods/184855ae-c7f2-40b8-af71-5df074137269/volumes"
Jul 16 22:59:43.602746 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:43.602733    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" path="/var/lib/kubelet/pods/d3cb4943-28d4-42ad-9589-4876b37c2117/volumes"
Jul 16 22:59:56.711192 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711161    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"]
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711484    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711495    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711503    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711508    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711518    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711523    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711533    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711538    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711545    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711549    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711555    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711559    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711565    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711569    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711577    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711581    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711588    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711593    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711598    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711603    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711610    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.711629 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711614    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711664    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711672    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711678    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711685    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7000c12b-0a24-4d0b-8147-12334bb4f30e" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711690    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711695    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711700    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711707    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.712347 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.711712    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3cb4943-28d4-42ad-9589-4876b37c2117" containerName="revoke-keys"
Jul 16 22:59:56.716463 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.716447    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.718836 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.718817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-25067845-openshift-default-dockercfg-8swrx\""
Jul 16 22:59:56.734068 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.734040    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"]
Jul 16 22:59:56.849079 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849044    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-data\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849087    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-token\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpd9p\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-kube-api-access-cpd9p\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-socket\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849302 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-envoy\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-credential-socket\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-certs\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-podinfo\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.849514 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.849441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istiod-ca-cert\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950361 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-data\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950361 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-token\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpd9p\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-kube-api-access-cpd9p\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-socket\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-envoy\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-credential-socket\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-certs\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-podinfo\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.950641 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istiod-ca-cert\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.951035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950798    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-data\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.951035 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.950861    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-socket\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.951176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.951083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-certs\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.951176 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.951116    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-credential-socket\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.951333 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.951315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istiod-ca-cert\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.952971 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.952951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-envoy\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.953084 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.953068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-podinfo\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.967055 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.966993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-token\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:56.968803 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:56.968786    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpd9p\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-kube-api-access-cpd9p\") pod \"e2e-mt-25067845-openshift-default-69889c9f79-65d9c\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") " pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:57.026478 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.026445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:57.165491 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.165400    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"]
Jul 16 22:59:57.168104 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:57.168072    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9f70d3_80b2_407c_98f8_3bf6f46b8c20.slice/crio-b6b40d24e40f2819afb65b854108133f3d5d8ea058967e953687cb2c978f3edc WatchSource:0}: Error finding container b6b40d24e40f2819afb65b854108133f3d5d8ea058967e953687cb2c978f3edc: Status 404 returned error can't find the container with id b6b40d24e40f2819afb65b854108133f3d5d8ea058967e953687cb2c978f3edc
Jul 16 22:59:57.170051 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.170020    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:59:57.170114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.170083    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:59:57.170114 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.170111    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 22:59:57.521520 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.521426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" event={"ID":"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20","Type":"ContainerStarted","Data":"4f359d49954fd7df5255c6c543cfd48fd4b97162a99566f9bbc1863ae14e8106"}
Jul 16 22:59:57.521520 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.521478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" event={"ID":"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20","Type":"ContainerStarted","Data":"b6b40d24e40f2819afb65b854108133f3d5d8ea058967e953687cb2c978f3edc"}
Jul 16 22:59:57.544015 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:57.543957    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podStartSLOduration=1.543940329 podStartE2EDuration="1.543940329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:59:57.541820434 +0000 UTC m=+2553.862376505" watchObservedRunningTime="2026-07-16 22:59:57.543940329 +0000 UTC m=+2553.864496401"
Jul 16 22:59:58.026957 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.026923    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:58.031882 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.031853    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:58.526604 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.526577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:58.527637 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.527619    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 22:59:58.801823 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.801740    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"]
Jul 16 22:59:58.802159 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.802145    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:58.802215 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.802161    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:58.802252 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.802240    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c640b184-4f44-4f9a-aaac-e04735e525c8" containerName="revoke-keys"
Jul 16 22:59:58.802303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.802252    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="184855ae-c7f2-40b8-af71-5df074137269" containerName="istio-proxy"
Jul 16 22:59:58.802303 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.802260    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94609380-bdab-4fc1-90db-ff09046c8b5e" containerName="revoke-keys"
Jul 16 22:59:58.805539 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.805519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 22:59:58.807900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.807876    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:59:58.807900 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.807889    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 22:59:58.813005 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.812973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"]
Jul 16 22:59:58.968895 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:58.968845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66003020-9aa8-46ce-9624-1e724b78ca4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rb8rt\" (UID: \"66003020-9aa8-46ce-9624-1e724b78ca4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 22:59:59.070166 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.070044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66003020-9aa8-46ce-9624-1e724b78ca4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rb8rt\" (UID: \"66003020-9aa8-46ce-9624-1e724b78ca4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 22:59:59.070685 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.070663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66003020-9aa8-46ce-9624-1e724b78ca4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rb8rt\" (UID: \"66003020-9aa8-46ce-9624-1e724b78ca4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 22:59:59.124341 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.124313    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 22:59:59.262535 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.262313    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"]
Jul 16 22:59:59.264703 ip-10-0-136-7 kubenswrapper[2564]: W0716 22:59:59.264674    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66003020_9aa8_46ce_9624_1e724b78ca4b.slice/crio-2e775dc44157d86312c3071e1f075b27abc0c3fac02c50de2ca0fa7efd68b4c2 WatchSource:0}: Error finding container 2e775dc44157d86312c3071e1f075b27abc0c3fac02c50de2ca0fa7efd68b4c2: Status 404 returned error can't find the container with id 2e775dc44157d86312c3071e1f075b27abc0c3fac02c50de2ca0fa7efd68b4c2
Jul 16 22:59:59.533471 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.533431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" event={"ID":"66003020-9aa8-46ce-9624-1e724b78ca4b","Type":"ContainerStarted","Data":"ad5a81fbad800c7c7b3703718d80ee5c7cb13281c528780a79555252a694cc94"}
Jul 16 22:59:59.533676 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.533487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" event={"ID":"66003020-9aa8-46ce-9624-1e724b78ca4b","Type":"ContainerStarted","Data":"2e775dc44157d86312c3071e1f075b27abc0c3fac02c50de2ca0fa7efd68b4c2"}
Jul 16 22:59:59.554977 ip-10-0-136-7 kubenswrapper[2564]: I0716 22:59:59.554918    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" podStartSLOduration=1.554901992 podStartE2EDuration="1.554901992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:59:59.552567571 +0000 UTC m=+2555.873123645" watchObservedRunningTime="2026-07-16 22:59:59.554901992 +0000 UTC m=+2555.875458064"
Jul 16 23:00:00.133412 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.133322    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"]
Jul 16 23:00:00.136373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.136354    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"]
Jul 16 23:00:00.136489 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.136468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"
Jul 16 23:00:00.139583 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.139563    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"
Jul 16 23:00:00.146795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.146673    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"]
Jul 16 23:00:00.149106 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.149085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"
Jul 16 23:00:00.150045 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.150021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"]
Jul 16 23:00:00.157944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.157925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"
Jul 16 23:00:00.283051 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.283013    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"]
Jul 16 23:00:00.286573 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.286551    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:00.301654 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.301626    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"]
Jul 16 23:00:00.327912 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.327883    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"]
Jul 16 23:00:00.329470 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:00:00.329440    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9534a6_15a8_4fae_b746_b7c2d39541bf.slice/crio-e746fff90d050cf9bcb475c41d5afd629f5de598dc091b484e2998cf22e62860 WatchSource:0}: Error finding container e746fff90d050cf9bcb475c41d5afd629f5de598dc091b484e2998cf22e62860: Status 404 returned error can't find the container with id e746fff90d050cf9bcb475c41d5afd629f5de598dc091b484e2998cf22e62860
Jul 16 23:00:00.382171 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.382117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b254dc3c-820f-4f1c-86c8-5ba734372cb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs\" (UID: \"b254dc3c-820f-4f1c-86c8-5ba734372cb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:00.483617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.483588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b254dc3c-820f-4f1c-86c8-5ba734372cb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs\" (UID: \"b254dc3c-820f-4f1c-86c8-5ba734372cb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:00.484205 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.484184    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b254dc3c-820f-4f1c-86c8-5ba734372cb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs\" (UID: \"b254dc3c-820f-4f1c-86c8-5ba734372cb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:00.516984 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.516961    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"]
Jul 16 23:00:00.518945 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:00:00.518916    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf737ccc9_fdb1_4086_b79f_bb26fff932e3.slice/crio-38711609a3670ef3ee5fa315ffce80e5a18416e18809e5d4cb92de445cf138d9 WatchSource:0}: Error finding container 38711609a3670ef3ee5fa315ffce80e5a18416e18809e5d4cb92de445cf138d9: Status 404 returned error can't find the container with id 38711609a3670ef3ee5fa315ffce80e5a18416e18809e5d4cb92de445cf138d9
Jul 16 23:00:00.542693 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.542655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" event={"ID":"f737ccc9-fdb1-4086-b79f-bb26fff932e3","Type":"ContainerStarted","Data":"38711609a3670ef3ee5fa315ffce80e5a18416e18809e5d4cb92de445cf138d9"}
Jul 16 23:00:00.544124 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.544098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7" event={"ID":"6f9534a6-15a8-4fae-b746-b7c2d39541bf","Type":"ContainerStarted","Data":"e746fff90d050cf9bcb475c41d5afd629f5de598dc091b484e2998cf22e62860"}
Jul 16 23:00:00.546122 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.546073    2564 generic.go:356] "Generic (PLEG): container finished" podID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerID="ad5a81fbad800c7c7b3703718d80ee5c7cb13281c528780a79555252a694cc94" exitCode=6
Jul 16 23:00:00.546770 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.546262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" event={"ID":"66003020-9aa8-46ce-9624-1e724b78ca4b","Type":"ContainerDied","Data":"ad5a81fbad800c7c7b3703718d80ee5c7cb13281c528780a79555252a694cc94"}
Jul 16 23:00:00.546770 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.546440    2564 scope.go:117] "RemoveContainer" containerID="ad5a81fbad800c7c7b3703718d80ee5c7cb13281c528780a79555252a694cc94"
Jul 16 23:00:00.599373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.599348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:00.738678 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:00.738653    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"]
Jul 16 23:00:00.740330 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:00:00.740301    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb254dc3c_820f_4f1c_86c8_5ba734372cb4.slice/crio-1ac7e6aa3b2409e895cdc8b16ab81f78914f8cf8e41ae2092c0d12c4f4493b93 WatchSource:0}: Error finding container 1ac7e6aa3b2409e895cdc8b16ab81f78914f8cf8e41ae2092c0d12c4f4493b93: Status 404 returned error can't find the container with id 1ac7e6aa3b2409e895cdc8b16ab81f78914f8cf8e41ae2092c0d12c4f4493b93
Jul 16 23:00:01.554436 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.554401    2564 generic.go:356] "Generic (PLEG): container finished" podID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerID="eeca2eb6d655ced3d56c54be2ed9b314565557a17d4a02b5fe83947a7b3cc927" exitCode=6
Jul 16 23:00:01.554977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.554495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" event={"ID":"66003020-9aa8-46ce-9624-1e724b78ca4b","Type":"ContainerDied","Data":"eeca2eb6d655ced3d56c54be2ed9b314565557a17d4a02b5fe83947a7b3cc927"}
Jul 16 23:00:01.554977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.554557    2564 scope.go:117] "RemoveContainer" containerID="ad5a81fbad800c7c7b3703718d80ee5c7cb13281c528780a79555252a694cc94"
Jul 16 23:00:01.554977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.554798    2564 scope.go:117] "RemoveContainer" containerID="eeca2eb6d655ced3d56c54be2ed9b314565557a17d4a02b5fe83947a7b3cc927"
Jul 16 23:00:01.555169 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:01.555054    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rb8rt_odh-ai-gateway-infra(66003020-9aa8-46ce-9624-1e724b78ca4b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b"
Jul 16 23:00:01.556078 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.556059    2564 generic.go:356] "Generic (PLEG): container finished" podID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerID="85c8da1683fd7c906ebe2dcedd305b49f11ff3e5d72ea73056ff32749bd3b366" exitCode=6
Jul 16 23:00:01.556201 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.556154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" event={"ID":"b254dc3c-820f-4f1c-86c8-5ba734372cb4","Type":"ContainerDied","Data":"85c8da1683fd7c906ebe2dcedd305b49f11ff3e5d72ea73056ff32749bd3b366"}
Jul 16 23:00:01.556201 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.556195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" event={"ID":"b254dc3c-820f-4f1c-86c8-5ba734372cb4","Type":"ContainerStarted","Data":"1ac7e6aa3b2409e895cdc8b16ab81f78914f8cf8e41ae2092c0d12c4f4493b93"}
Jul 16 23:00:01.556328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.556309    2564 scope.go:117] "RemoveContainer" containerID="85c8da1683fd7c906ebe2dcedd305b49f11ff3e5d72ea73056ff32749bd3b366"
Jul 16 23:00:01.557890 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.557864    2564 generic.go:356] "Generic (PLEG): container finished" podID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerID="5b52ed183ca0945bb203eb23e1e6604de72bb0becc36564f1c2fbeb1c7e22af2" exitCode=7
Jul 16 23:00:01.558008 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.557897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" event={"ID":"f737ccc9-fdb1-4086-b79f-bb26fff932e3","Type":"ContainerDied","Data":"5b52ed183ca0945bb203eb23e1e6604de72bb0becc36564f1c2fbeb1c7e22af2"}
Jul 16 23:00:01.558093 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.558076    2564 scope.go:117] "RemoveContainer" containerID="5b52ed183ca0945bb203eb23e1e6604de72bb0becc36564f1c2fbeb1c7e22af2"
Jul 16 23:00:01.559845 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.559800    2564 generic.go:356] "Generic (PLEG): container finished" podID="6f9534a6-15a8-4fae-b746-b7c2d39541bf" containerID="5845f28c83e6a7ac1574c37e795dda84cbae39fd2ddd23e868657514604859f1" exitCode=0
Jul 16 23:00:01.559920 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:01.559856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7" event={"ID":"6f9534a6-15a8-4fae-b746-b7c2d39541bf","Type":"ContainerDied","Data":"5845f28c83e6a7ac1574c37e795dda84cbae39fd2ddd23e868657514604859f1"}
Jul 16 23:00:02.567739 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.567711    2564 scope.go:117] "RemoveContainer" containerID="eeca2eb6d655ced3d56c54be2ed9b314565557a17d4a02b5fe83947a7b3cc927"
Jul 16 23:00:02.568182 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:02.567947    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rb8rt_odh-ai-gateway-infra(66003020-9aa8-46ce-9624-1e724b78ca4b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b"
Jul 16 23:00:02.569275 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.569252    2564 generic.go:356] "Generic (PLEG): container finished" podID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerID="66d4ff243c38c3820499766cdf1805c3ad5b5fce247a0dab7fa5b7bccda402d2" exitCode=6
Jul 16 23:00:02.569366 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.569284    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" event={"ID":"b254dc3c-820f-4f1c-86c8-5ba734372cb4","Type":"ContainerDied","Data":"66d4ff243c38c3820499766cdf1805c3ad5b5fce247a0dab7fa5b7bccda402d2"}
Jul 16 23:00:02.569366 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.569334    2564 scope.go:117] "RemoveContainer" containerID="85c8da1683fd7c906ebe2dcedd305b49f11ff3e5d72ea73056ff32749bd3b366"
Jul 16 23:00:02.569479 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.569464    2564 scope.go:117] "RemoveContainer" containerID="66d4ff243c38c3820499766cdf1805c3ad5b5fce247a0dab7fa5b7bccda402d2"
Jul 16 23:00:02.569681 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:02.569659    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs_odh-ai-gateway-infra(b254dc3c-820f-4f1c-86c8-5ba734372cb4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4"
Jul 16 23:00:02.571080 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.571062    2564 generic.go:356] "Generic (PLEG): container finished" podID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerID="44adbbf02b89fb9191c3fe357d59e72b5171dfd5fa28fcfddf54f680e0a8244a" exitCode=7
Jul 16 23:00:02.571183 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.571112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" event={"ID":"f737ccc9-fdb1-4086-b79f-bb26fff932e3","Type":"ContainerDied","Data":"44adbbf02b89fb9191c3fe357d59e72b5171dfd5fa28fcfddf54f680e0a8244a"}
Jul 16 23:00:02.571274 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.571259    2564 scope.go:117] "RemoveContainer" containerID="44adbbf02b89fb9191c3fe357d59e72b5171dfd5fa28fcfddf54f680e0a8244a"
Jul 16 23:00:02.571447 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:02.571432    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz_odh-ai-gateway-infra(f737ccc9-fdb1-4086-b79f-bb26fff932e3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3"
Jul 16 23:00:02.580468 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.580451    2564 scope.go:117] "RemoveContainer" containerID="5b52ed183ca0945bb203eb23e1e6604de72bb0becc36564f1c2fbeb1c7e22af2"
Jul 16 23:00:02.705701 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:02.705676    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"
Jul 16 23:00:03.577815 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:03.577782    2564 scope.go:117] "RemoveContainer" containerID="66d4ff243c38c3820499766cdf1805c3ad5b5fce247a0dab7fa5b7bccda402d2"
Jul 16 23:00:03.578253 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:03.578029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs_odh-ai-gateway-infra(b254dc3c-820f-4f1c-86c8-5ba734372cb4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4"
Jul 16 23:00:03.579209 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:03.579189    2564 scope.go:117] "RemoveContainer" containerID="44adbbf02b89fb9191c3fe357d59e72b5171dfd5fa28fcfddf54f680e0a8244a"
Jul 16 23:00:03.579444 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:03.579417    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz_odh-ai-gateway-infra(f737ccc9-fdb1-4086-b79f-bb26fff932e3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3"
Jul 16 23:00:03.580240 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:03.580224    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7"
Jul 16 23:00:03.580349 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:03.580255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-t5cg7" event={"ID":"6f9534a6-15a8-4fae-b746-b7c2d39541bf","Type":"ContainerDied","Data":"e746fff90d050cf9bcb475c41d5afd629f5de598dc091b484e2998cf22e62860"}
Jul 16 23:00:03.580349 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:03.580290    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e746fff90d050cf9bcb475c41d5afd629f5de598dc091b484e2998cf22e62860"
Jul 16 23:00:06.414989 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:06.414950    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:00:12.263384 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.263337    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"]
Jul 16 23:00:12.263919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.263863    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9534a6-15a8-4fae-b746-b7c2d39541bf" containerName="cleanup"
Jul 16 23:00:12.263919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.263885    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9534a6-15a8-4fae-b746-b7c2d39541bf" containerName="cleanup"
Jul 16 23:00:12.264037 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.264013    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9534a6-15a8-4fae-b746-b7c2d39541bf" containerName="cleanup"
Jul 16 23:00:12.267388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.267365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:12.276813 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.276785    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"]
Jul 16 23:00:12.300312 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.300272    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs\" (UID: \"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:12.401897 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.401848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs\" (UID: \"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:12.402665 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.402641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs\" (UID: \"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:12.581002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.580913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:12.720013 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:12.719983    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"]
Jul 16 23:00:12.721810 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:00:12.721778    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17eda9d3_e1bd_46ee_b91e_3fa2e15826fe.slice/crio-ec0ebe42af3f280e715eb92cf3da49c3f6e9560aed74191008f615c02cb605d9 WatchSource:0}: Error finding container ec0ebe42af3f280e715eb92cf3da49c3f6e9560aed74191008f615c02cb605d9: Status 404 returned error can't find the container with id ec0ebe42af3f280e715eb92cf3da49c3f6e9560aed74191008f615c02cb605d9
Jul 16 23:00:13.637809 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:13.637715    2564 generic.go:356] "Generic (PLEG): container finished" podID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerID="69dce2a1b7556e2936e2b4263063b1eff7c8e75294e9b3939ef39b977da36240" exitCode=6
Jul 16 23:00:13.638318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:13.637802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" event={"ID":"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe","Type":"ContainerDied","Data":"69dce2a1b7556e2936e2b4263063b1eff7c8e75294e9b3939ef39b977da36240"}
Jul 16 23:00:13.638318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:13.637861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" event={"ID":"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe","Type":"ContainerStarted","Data":"ec0ebe42af3f280e715eb92cf3da49c3f6e9560aed74191008f615c02cb605d9"}
Jul 16 23:00:13.638318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:13.638073    2564 scope.go:117] "RemoveContainer" containerID="69dce2a1b7556e2936e2b4263063b1eff7c8e75294e9b3939ef39b977da36240"
Jul 16 23:00:14.053566 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.053520    2564 patch_prober.go:28] interesting pod/e2e-mt-25067845-openshift-default-69889c9f79-65d9c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": read tcp 10.134.0.2:45722->10.134.0.100:15021: read: connection reset by peer" start-of-body=
Jul 16 23:00:14.053733 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.053598    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": read tcp 10.134.0.2:45722->10.134.0.100:15021: read: connection reset by peer"
Jul 16 23:00:14.596345 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.596308    2564 scope.go:117] "RemoveContainer" containerID="44adbbf02b89fb9191c3fe357d59e72b5171dfd5fa28fcfddf54f680e0a8244a"
Jul 16 23:00:14.645233 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.645197    2564 generic.go:356] "Generic (PLEG): container finished" podID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerID="4f359d49954fd7df5255c6c543cfd48fd4b97162a99566f9bbc1863ae14e8106" exitCode=0
Jul 16 23:00:14.645629 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.645266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" event={"ID":"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20","Type":"ContainerDied","Data":"4f359d49954fd7df5255c6c543cfd48fd4b97162a99566f9bbc1863ae14e8106"}
Jul 16 23:00:14.645690 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.645674    2564 scope.go:117] "RemoveContainer" containerID="4f359d49954fd7df5255c6c543cfd48fd4b97162a99566f9bbc1863ae14e8106"
Jul 16 23:00:14.646761 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.646728    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:00:14.646878 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.646824    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:00:14.646937 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.646892    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:00:14.647315 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.647295    2564 generic.go:356] "Generic (PLEG): container finished" podID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerID="4d5ffd1bc80c03270ff32118fa4cd340f193668ec77ee3496c855e4cfa4494b8" exitCode=6
Jul 16 23:00:14.647368 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.647326    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" event={"ID":"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe","Type":"ContainerDied","Data":"4d5ffd1bc80c03270ff32118fa4cd340f193668ec77ee3496c855e4cfa4494b8"}
Jul 16 23:00:14.647368 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.647363    2564 scope.go:117] "RemoveContainer" containerID="69dce2a1b7556e2936e2b4263063b1eff7c8e75294e9b3939ef39b977da36240"
Jul 16 23:00:14.647517 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:14.647504    2564 scope.go:117] "RemoveContainer" containerID="4d5ffd1bc80c03270ff32118fa4cd340f193668ec77ee3496c855e4cfa4494b8"
Jul 16 23:00:14.647724 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:14.647701    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs_odh-ai-gateway-infra(17eda9d3-e1bd-46ee-b91e-3fa2e15826fe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe"
Jul 16 23:00:15.185851 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:15.185816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:00:15.655577 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:15.655544    2564 scope.go:117] "RemoveContainer" containerID="4d5ffd1bc80c03270ff32118fa4cd340f193668ec77ee3496c855e4cfa4494b8"
Jul 16 23:00:15.656007 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:15.655761    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs_odh-ai-gateway-infra(17eda9d3-e1bd-46ee-b91e-3fa2e15826fe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe"
Jul 16 23:00:15.657239 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:15.657201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" event={"ID":"f737ccc9-fdb1-4086-b79f-bb26fff932e3","Type":"ContainerStarted","Data":"3aa884d82e987a3ec2a67f5c4162f7402b6a81f9f1b60b7de1b5332e2f6aec94"}
Jul 16 23:00:15.659094 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:15.659070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" event={"ID":"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20","Type":"ContainerStarted","Data":"7ec436e3d86503e0ae106b6c9630f13bd2b907091bfe6c6906129842348f6be2"}
Jul 16 23:00:15.721949 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:15.721874    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" podStartSLOduration=15.721854487 podStartE2EDuration="15.721854487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:15.717777656 +0000 UTC m=+2572.038333727" watchObservedRunningTime="2026-07-16 23:00:15.721854487 +0000 UTC m=+2572.042410559"
Jul 16 23:00:16.026762 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.026724    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 23:00:16.027002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.026978    2564 patch_prober.go:28] interesting pod/e2e-mt-25067845-openshift-default-69889c9f79-65d9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Jul 16 23:00:16.027057 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.027039    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Jul 16 23:00:16.665932 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.665899    2564 generic.go:356] "Generic (PLEG): container finished" podID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerID="3aa884d82e987a3ec2a67f5c4162f7402b6a81f9f1b60b7de1b5332e2f6aec94" exitCode=0
Jul 16 23:00:16.666435 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.665973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" event={"ID":"f737ccc9-fdb1-4086-b79f-bb26fff932e3","Type":"ContainerDied","Data":"3aa884d82e987a3ec2a67f5c4162f7402b6a81f9f1b60b7de1b5332e2f6aec94"}
Jul 16 23:00:16.666435 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.666048    2564 scope.go:117] "RemoveContainer" containerID="44adbbf02b89fb9191c3fe357d59e72b5171dfd5fa28fcfddf54f680e0a8244a"
Jul 16 23:00:16.729211 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:16.729179    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"]
Jul 16 23:00:17.027396 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:17.027358    2564 patch_prober.go:28] interesting pod/e2e-mt-25067845-openshift-default-69889c9f79-65d9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Jul 16 23:00:17.027580 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:17.027427    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Jul 16 23:00:17.596723 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:17.596691    2564 scope.go:117] "RemoveContainer" containerID="eeca2eb6d655ced3d56c54be2ed9b314565557a17d4a02b5fe83947a7b3cc927"
Jul 16 23:00:17.597192 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:17.597171    2564 scope.go:117] "RemoveContainer" containerID="66d4ff243c38c3820499766cdf1805c3ad5b5fce247a0dab7fa5b7bccda402d2"
Jul 16 23:00:17.911963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:17.911941    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"
Jul 16 23:00:18.027717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.027681    2564 patch_prober.go:28] interesting pod/e2e-mt-25067845-openshift-default-69889c9f79-65d9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Jul 16 23:00:18.027877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.027733    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Jul 16 23:00:18.689298 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.689255    2564 generic.go:356] "Generic (PLEG): container finished" podID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerID="d14c1714d545b2300043b40d185f7964d1e54b6f1e0393c1ba4af5978e286b3a" exitCode=6
Jul 16 23:00:18.689493 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.689354    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" event={"ID":"66003020-9aa8-46ce-9624-1e724b78ca4b","Type":"ContainerDied","Data":"d14c1714d545b2300043b40d185f7964d1e54b6f1e0393c1ba4af5978e286b3a"}
Jul 16 23:00:18.689493 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.689408    2564 scope.go:117] "RemoveContainer" containerID="eeca2eb6d655ced3d56c54be2ed9b314565557a17d4a02b5fe83947a7b3cc927"
Jul 16 23:00:18.689800 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.689775    2564 scope.go:117] "RemoveContainer" containerID="d14c1714d545b2300043b40d185f7964d1e54b6f1e0393c1ba4af5978e286b3a"
Jul 16 23:00:18.690073 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:18.690045    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rb8rt_odh-ai-gateway-infra(66003020-9aa8-46ce-9624-1e724b78ca4b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b"
Jul 16 23:00:18.694115 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.694083    2564 generic.go:356] "Generic (PLEG): container finished" podID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerID="3b2aa5ae0e3f1ce857a8c76d7e15efe99dd2df7dc39e0731330f7e2a594cc264" exitCode=6
Jul 16 23:00:18.694244 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.694191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" event={"ID":"b254dc3c-820f-4f1c-86c8-5ba734372cb4","Type":"ContainerDied","Data":"3b2aa5ae0e3f1ce857a8c76d7e15efe99dd2df7dc39e0731330f7e2a594cc264"}
Jul 16 23:00:18.694822 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.694545    2564 scope.go:117] "RemoveContainer" containerID="3b2aa5ae0e3f1ce857a8c76d7e15efe99dd2df7dc39e0731330f7e2a594cc264"
Jul 16 23:00:18.694822 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:18.694769    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs_odh-ai-gateway-infra(b254dc3c-820f-4f1c-86c8-5ba734372cb4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4"
Jul 16 23:00:18.696365 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.696336    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"
Jul 16 23:00:18.696476 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.696425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz" event={"ID":"f737ccc9-fdb1-4086-b79f-bb26fff932e3","Type":"ContainerDied","Data":"38711609a3670ef3ee5fa315ffce80e5a18416e18809e5d4cb92de445cf138d9"}
Jul 16 23:00:18.696476 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.696471    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38711609a3670ef3ee5fa315ffce80e5a18416e18809e5d4cb92de445cf138d9"
Jul 16 23:00:18.705300 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.705285    2564 scope.go:117] "RemoveContainer" containerID="66d4ff243c38c3820499766cdf1805c3ad5b5fce247a0dab7fa5b7bccda402d2"
Jul 16 23:00:18.756332 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.756252    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"]
Jul 16 23:00:18.760475 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:18.760450    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-25067845-29737380-rstlz"]
Jul 16 23:00:19.026852 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.026753    2564 patch_prober.go:28] interesting pod/e2e-mt-25067845-openshift-default-69889c9f79-65d9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Jul 16 23:00:19.026852 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.026822    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Jul 16 23:00:19.601309 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.601268    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" path="/var/lib/kubelet/pods/f737ccc9-fdb1-4086-b79f-bb26fff932e3/volumes"
Jul 16 23:00:19.717476 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.717443    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"]
Jul 16 23:00:19.733712 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.733672    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"]
Jul 16 23:00:19.858994 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.858908    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 23:00:19.900859 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.900833    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:19.972735 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.972698    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b254dc3c-820f-4f1c-86c8-5ba734372cb4-maas-api-service-ca\") pod \"b254dc3c-820f-4f1c-86c8-5ba734372cb4\" (UID: \"b254dc3c-820f-4f1c-86c8-5ba734372cb4\") "
Jul 16 23:00:19.972883 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.972809    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66003020-9aa8-46ce-9624-1e724b78ca4b-maas-api-service-ca\") pod \"66003020-9aa8-46ce-9624-1e724b78ca4b\" (UID: \"66003020-9aa8-46ce-9624-1e724b78ca4b\") "
Jul 16 23:00:19.973049 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.973028    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b254dc3c-820f-4f1c-86c8-5ba734372cb4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b254dc3c-820f-4f1c-86c8-5ba734372cb4" (UID: "b254dc3c-820f-4f1c-86c8-5ba734372cb4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:00:19.973205 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:19.973185    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66003020-9aa8-46ce-9624-1e724b78ca4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66003020-9aa8-46ce-9624-1e724b78ca4b" (UID: "66003020-9aa8-46ce-9624-1e724b78ca4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:00:20.030459 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.030430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 23:00:20.030842 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.030683    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 23:00:20.031449 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.031434    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 23:00:20.074033 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.073986    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b254dc3c-820f-4f1c-86c8-5ba734372cb4-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:00:20.074033 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.074017    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66003020-9aa8-46ce-9624-1e724b78ca4b-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:00:20.712324 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.712287    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"
Jul 16 23:00:20.712510 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.712290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt" event={"ID":"66003020-9aa8-46ce-9624-1e724b78ca4b","Type":"ContainerDied","Data":"2e775dc44157d86312c3071e1f075b27abc0c3fac02c50de2ca0fa7efd68b4c2"}
Jul 16 23:00:20.712510 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.712424    2564 scope.go:117] "RemoveContainer" containerID="d14c1714d545b2300043b40d185f7964d1e54b6f1e0393c1ba4af5978e286b3a"
Jul 16 23:00:20.713850 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.713830    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"
Jul 16 23:00:20.713980 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.713871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs" event={"ID":"b254dc3c-820f-4f1c-86c8-5ba734372cb4","Type":"ContainerDied","Data":"1ac7e6aa3b2409e895cdc8b16ab81f78914f8cf8e41ae2092c0d12c4f4493b93"}
Jul 16 23:00:20.724712 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.724652    2564 scope.go:117] "RemoveContainer" containerID="3b2aa5ae0e3f1ce857a8c76d7e15efe99dd2df7dc39e0731330f7e2a594cc264"
Jul 16 23:00:20.761607 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.761557    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"]
Jul 16 23:00:20.764055 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.764026    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb8rt"]
Jul 16 23:00:20.792575 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.792542    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"]
Jul 16 23:00:20.801199 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:20.801173    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-mgxzs"]
Jul 16 23:00:21.602444 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:21.602401    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" path="/var/lib/kubelet/pods/66003020-9aa8-46ce-9624-1e724b78ca4b/volumes"
Jul 16 23:00:21.602934 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:21.602918    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" path="/var/lib/kubelet/pods/b254dc3c-820f-4f1c-86c8-5ba734372cb4/volumes"
Jul 16 23:00:24.509252 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:24.509219    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:00:26.596383 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.596351    2564 scope.go:117] "RemoveContainer" containerID="4d5ffd1bc80c03270ff32118fa4cd340f193668ec77ee3496c855e4cfa4494b8"
Jul 16 23:00:26.793423 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793397    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"]
Jul 16 23:00:26.793755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793742    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.793796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793757    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.793796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793773    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.793796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793781    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.793796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793793    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793799    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793810    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793815    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793824    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793828    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793835    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793840    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793852    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793857    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793902    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.793918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793914    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.794266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793923    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:26.794266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793931    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.794266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793936    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.794266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793942    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.794266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793948    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:26.794266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.793953    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:26.796572 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.796555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:26.815147 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.815113    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"]
Jul 16 23:00:26.838190 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.838121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b336b9dc-0efd-422a-b32e-cc2cd25d3e64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-25067845-sxrsd\" (UID: \"b336b9dc-0efd-422a-b32e-cc2cd25d3e64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:26.938759 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.938720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b336b9dc-0efd-422a-b32e-cc2cd25d3e64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-25067845-sxrsd\" (UID: \"b336b9dc-0efd-422a-b32e-cc2cd25d3e64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:26.939342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:26.939323    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b336b9dc-0efd-422a-b32e-cc2cd25d3e64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-25067845-sxrsd\" (UID: \"b336b9dc-0efd-422a-b32e-cc2cd25d3e64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:27.106243 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.106205    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:27.243262 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.243237    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"]
Jul 16 23:00:27.244993 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:00:27.244964    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb336b9dc_0efd_422a_b32e_cc2cd25d3e64.slice/crio-ca9d4cc9445e67baf3e3c95d1089e09913871cc8f208dfaeb0bec8be63781981 WatchSource:0}: Error finding container ca9d4cc9445e67baf3e3c95d1089e09913871cc8f208dfaeb0bec8be63781981: Status 404 returned error can't find the container with id ca9d4cc9445e67baf3e3c95d1089e09913871cc8f208dfaeb0bec8be63781981
Jul 16 23:00:27.755796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.755688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd" event={"ID":"b336b9dc-0efd-422a-b32e-cc2cd25d3e64","Type":"ContainerStarted","Data":"19d756ad1e8c18d1eb015fc73d6402b00e293aab1fd94758e025bf13af110620"}
Jul 16 23:00:27.755796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.755751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd" event={"ID":"b336b9dc-0efd-422a-b32e-cc2cd25d3e64","Type":"ContainerStarted","Data":"ca9d4cc9445e67baf3e3c95d1089e09913871cc8f208dfaeb0bec8be63781981"}
Jul 16 23:00:27.757483 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.757459    2564 generic.go:356] "Generic (PLEG): container finished" podID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerID="de4a5e65b65f0a7607c2520606e64176fe477fb69bb59ae1e7c144e54d771fca" exitCode=6
Jul 16 23:00:27.757577 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.757503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" event={"ID":"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe","Type":"ContainerDied","Data":"de4a5e65b65f0a7607c2520606e64176fe477fb69bb59ae1e7c144e54d771fca"}
Jul 16 23:00:27.757577 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.757551    2564 scope.go:117] "RemoveContainer" containerID="4d5ffd1bc80c03270ff32118fa4cd340f193668ec77ee3496c855e4cfa4494b8"
Jul 16 23:00:27.757834 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.757812    2564 scope.go:117] "RemoveContainer" containerID="de4a5e65b65f0a7607c2520606e64176fe477fb69bb59ae1e7c144e54d771fca"
Jul 16 23:00:27.758055 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:00:27.758035    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs_odh-ai-gateway-infra(17eda9d3-e1bd-46ee-b91e-3fa2e15826fe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe"
Jul 16 23:00:27.773779 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:27.773731    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd" podStartSLOduration=1.773714776 podStartE2EDuration="1.773714776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:27.772858371 +0000 UTC m=+2584.093414435" watchObservedRunningTime="2026-07-16 23:00:27.773714776 +0000 UTC m=+2584.094270866"
Jul 16 23:00:28.765002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:28.764961    2564 generic.go:356] "Generic (PLEG): container finished" podID="b336b9dc-0efd-422a-b32e-cc2cd25d3e64" containerID="19d756ad1e8c18d1eb015fc73d6402b00e293aab1fd94758e025bf13af110620" exitCode=0
Jul 16 23:00:28.765379 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:28.765038    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd" event={"ID":"b336b9dc-0efd-422a-b32e-cc2cd25d3e64","Type":"ContainerDied","Data":"19d756ad1e8c18d1eb015fc73d6402b00e293aab1fd94758e025bf13af110620"}
Jul 16 23:00:28.806937 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:28.806895    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"]
Jul 16 23:00:28.952816 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:28.952795    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:29.060880 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.060801    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe-maas-api-service-ca\") pod \"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe\" (UID: \"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe\") "
Jul 16 23:00:29.061192 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.061172    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" (UID: "17eda9d3-e1bd-46ee-b91e-3fa2e15826fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:00:29.162032 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.161997    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:00:29.772196 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.772165    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"
Jul 16 23:00:29.772196 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.772189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs" event={"ID":"17eda9d3-e1bd-46ee-b91e-3fa2e15826fe","Type":"ContainerDied","Data":"ec0ebe42af3f280e715eb92cf3da49c3f6e9560aed74191008f615c02cb605d9"}
Jul 16 23:00:29.772731 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.772252    2564 scope.go:117] "RemoveContainer" containerID="de4a5e65b65f0a7607c2520606e64176fe477fb69bb59ae1e7c144e54d771fca"
Jul 16 23:00:29.789957 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.789928    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"]
Jul 16 23:00:29.802987 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.802320    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-sldhs"]
Jul 16 23:00:29.911927 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.911899    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:29.969478 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.969439    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b336b9dc-0efd-422a-b32e-cc2cd25d3e64-maas-api-service-ca\") pod \"b336b9dc-0efd-422a-b32e-cc2cd25d3e64\" (UID: \"b336b9dc-0efd-422a-b32e-cc2cd25d3e64\") "
Jul 16 23:00:29.969834 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:29.969813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b336b9dc-0efd-422a-b32e-cc2cd25d3e64-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b336b9dc-0efd-422a-b32e-cc2cd25d3e64" (UID: "b336b9dc-0efd-422a-b32e-cc2cd25d3e64"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:00:30.070699 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:30.070659    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b336b9dc-0efd-422a-b32e-cc2cd25d3e64-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:00:30.780571 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:30.780543    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd"
Jul 16 23:00:30.780954 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:30.780579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25067845-sxrsd" event={"ID":"b336b9dc-0efd-422a-b32e-cc2cd25d3e64","Type":"ContainerDied","Data":"ca9d4cc9445e67baf3e3c95d1089e09913871cc8f208dfaeb0bec8be63781981"}
Jul 16 23:00:30.780954 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:30.780621    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca9d4cc9445e67baf3e3c95d1089e09913871cc8f208dfaeb0bec8be63781981"
Jul 16 23:00:31.601951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:31.601918    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" path="/var/lib/kubelet/pods/17eda9d3-e1bd-46ee-b91e-3fa2e15826fe/volumes"
Jul 16 23:00:40.806287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:40.806251    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:00:58.901419 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901381    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"]
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901735    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901746    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f737ccc9-fdb1-4086-b79f-bb26fff932e3" containerName="cleanup"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901759    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901765    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901771    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901776    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901789    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901794    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b254dc3c-820f-4f1c-86c8-5ba734372cb4" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901802    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b336b9dc-0efd-422a-b32e-cc2cd25d3e64" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901808    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b336b9dc-0efd-422a-b32e-cc2cd25d3e64" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901866    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66003020-9aa8-46ce-9624-1e724b78ca4b" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901874    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901881    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901886    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:00:58.901944 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.901892    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b336b9dc-0efd-422a-b32e-cc2cd25d3e64" containerName="revoke-keys"
Jul 16 23:00:58.906021 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.906004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:00:58.915718 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.915682    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:00:58.915833 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.915810    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:00:58.933617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:58.933531    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"]
Jul 16 23:00:59.017162 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.017108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc039722-05b8-47af-8e29-017147cc528d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-mz7q8\" (UID: \"bc039722-05b8-47af-8e29-017147cc528d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:00:59.118320 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.118278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc039722-05b8-47af-8e29-017147cc528d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-mz7q8\" (UID: \"bc039722-05b8-47af-8e29-017147cc528d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:00:59.118977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.118955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc039722-05b8-47af-8e29-017147cc528d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-mz7q8\" (UID: \"bc039722-05b8-47af-8e29-017147cc528d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:00:59.215638 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.215603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:00:59.347382 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.347353    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"]
Jul 16 23:00:59.348587 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:00:59.348564    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc039722_05b8_47af_8e29_017147cc528d.slice/crio-9e4f7fb390dcf5b0f7a05f65421ee6df31649923c5ea82f77eaf38635fec2e25 WatchSource:0}: Error finding container 9e4f7fb390dcf5b0f7a05f65421ee6df31649923c5ea82f77eaf38635fec2e25: Status 404 returned error can't find the container with id 9e4f7fb390dcf5b0f7a05f65421ee6df31649923c5ea82f77eaf38635fec2e25
Jul 16 23:00:59.945923 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.945891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerStarted","Data":"85459102a01f1b3860f20040e837ba5c7fe811ff3f2296a98ad2c9f2e2730084"}
Jul 16 23:00:59.946336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.945951    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerStarted","Data":"9e4f7fb390dcf5b0f7a05f65421ee6df31649923c5ea82f77eaf38635fec2e25"}
Jul 16 23:00:59.962821 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:00:59.962763    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" podStartSLOduration=1.9627450579999999 podStartE2EDuration="1.962745058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:59.96197279 +0000 UTC m=+2616.282528865" watchObservedRunningTime="2026-07-16 23:00:59.962745058 +0000 UTC m=+2616.283301161"
Jul 16 23:01:00.952639 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:00.952606    2564 generic.go:356] "Generic (PLEG): container finished" podID="bc039722-05b8-47af-8e29-017147cc528d" containerID="85459102a01f1b3860f20040e837ba5c7fe811ff3f2296a98ad2c9f2e2730084" exitCode=6
Jul 16 23:01:00.953009 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:00.952699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerDied","Data":"85459102a01f1b3860f20040e837ba5c7fe811ff3f2296a98ad2c9f2e2730084"}
Jul 16 23:01:00.953009 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:00.952858    2564 scope.go:117] "RemoveContainer" containerID="85459102a01f1b3860f20040e837ba5c7fe811ff3f2296a98ad2c9f2e2730084"
Jul 16 23:01:01.959926 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:01.959894    2564 generic.go:356] "Generic (PLEG): container finished" podID="bc039722-05b8-47af-8e29-017147cc528d" containerID="cbacab14e6ef2232093149594cee35c86295b125c21bcd3c46121cbc54b5e8f3" exitCode=6
Jul 16 23:01:01.960379 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:01.959972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerDied","Data":"cbacab14e6ef2232093149594cee35c86295b125c21bcd3c46121cbc54b5e8f3"}
Jul 16 23:01:01.960379 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:01.960029    2564 scope.go:117] "RemoveContainer" containerID="85459102a01f1b3860f20040e837ba5c7fe811ff3f2296a98ad2c9f2e2730084"
Jul 16 23:01:01.960379 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:01.960284    2564 scope.go:117] "RemoveContainer" containerID="cbacab14e6ef2232093149594cee35c86295b125c21bcd3c46121cbc54b5e8f3"
Jul 16 23:01:01.960521 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:01.960501    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-mz7q8_odh-ai-gateway-infra(bc039722-05b8-47af-8e29-017147cc528d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" podUID="bc039722-05b8-47af-8e29-017147cc528d"
Jul 16 23:01:02.331477 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.331382    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"]
Jul 16 23:01:02.331833 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.331814    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:01:02.331916 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.331835    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17eda9d3-e1bd-46ee-b91e-3fa2e15826fe" containerName="revoke-keys"
Jul 16 23:01:02.335938 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.335917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:02.357331 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.357302    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"]
Jul 16 23:01:02.450742 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.450708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq\" (UID: \"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:02.552157 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.552099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq\" (UID: \"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:02.552724 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.552704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq\" (UID: \"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:02.646336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.646246    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:02.785025 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.784996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"]
Jul 16 23:01:02.787118 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:01:02.787089    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb855cee6_e63c_451e_b1b1_0eb9a63bd0c7.slice/crio-6d1e4e73acad2025f6785dc74f837135bcc22a9e7174f76004227e35ffd7f5a3 WatchSource:0}: Error finding container 6d1e4e73acad2025f6785dc74f837135bcc22a9e7174f76004227e35ffd7f5a3: Status 404 returned error can't find the container with id 6d1e4e73acad2025f6785dc74f837135bcc22a9e7174f76004227e35ffd7f5a3
Jul 16 23:01:02.967804 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.967775    2564 scope.go:117] "RemoveContainer" containerID="cbacab14e6ef2232093149594cee35c86295b125c21bcd3c46121cbc54b5e8f3"
Jul 16 23:01:02.968243 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:02.968032    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-mz7q8_odh-ai-gateway-infra(bc039722-05b8-47af-8e29-017147cc528d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" podUID="bc039722-05b8-47af-8e29-017147cc528d"
Jul 16 23:01:02.968794 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:02.968770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" event={"ID":"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7","Type":"ContainerStarted","Data":"6d1e4e73acad2025f6785dc74f837135bcc22a9e7174f76004227e35ffd7f5a3"}
Jul 16 23:01:03.975975 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:03.975933    2564 generic.go:356] "Generic (PLEG): container finished" podID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerID="4e65381f9b810949c6c9be2988efb3a688426ad6f7a4266de8ab494ec76617dd" exitCode=6
Jul 16 23:01:03.976409 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:03.976026    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" event={"ID":"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7","Type":"ContainerDied","Data":"4e65381f9b810949c6c9be2988efb3a688426ad6f7a4266de8ab494ec76617dd"}
Jul 16 23:01:03.976409 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:03.976218    2564 scope.go:117] "RemoveContainer" containerID="4e65381f9b810949c6c9be2988efb3a688426ad6f7a4266de8ab494ec76617dd"
Jul 16 23:01:04.983806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:04.983777    2564 generic.go:356] "Generic (PLEG): container finished" podID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerID="63f151bf65ae2ac2f411c06bc2203ed5bb07b59f1d1279645422db3865cfe2ed" exitCode=6
Jul 16 23:01:04.984301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:04.983829    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" event={"ID":"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7","Type":"ContainerDied","Data":"63f151bf65ae2ac2f411c06bc2203ed5bb07b59f1d1279645422db3865cfe2ed"}
Jul 16 23:01:04.984301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:04.983874    2564 scope.go:117] "RemoveContainer" containerID="4e65381f9b810949c6c9be2988efb3a688426ad6f7a4266de8ab494ec76617dd"
Jul 16 23:01:04.984301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:04.984047    2564 scope.go:117] "RemoveContainer" containerID="63f151bf65ae2ac2f411c06bc2203ed5bb07b59f1d1279645422db3865cfe2ed"
Jul 16 23:01:04.984487 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:04.984304    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq_odh-ai-gateway-infra(b855cee6-e63c-451e-b1b1-0eb9a63bd0c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7"
Jul 16 23:01:05.990878 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:05.990851    2564 scope.go:117] "RemoveContainer" containerID="63f151bf65ae2ac2f411c06bc2203ed5bb07b59f1d1279645422db3865cfe2ed"
Jul 16 23:01:05.991282 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:05.991076    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq_odh-ai-gateway-infra(b855cee6-e63c-451e-b1b1-0eb9a63bd0c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7"
Jul 16 23:01:15.599509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:15.599474    2564 scope.go:117] "RemoveContainer" containerID="cbacab14e6ef2232093149594cee35c86295b125c21bcd3c46121cbc54b5e8f3"
Jul 16 23:01:16.058596 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:16.058562    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerStarted","Data":"9c0ff6fb123870b20687e3dc26a47c364bf7abb2cbc2acf33fcfa35be0bf9da0"}
Jul 16 23:01:17.004094 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.004058    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"]
Jul 16 23:01:17.007433 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.007418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:17.016102 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.016079    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"]
Jul 16 23:01:17.065222 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.065186    2564 generic.go:356] "Generic (PLEG): container finished" podID="bc039722-05b8-47af-8e29-017147cc528d" containerID="9c0ff6fb123870b20687e3dc26a47c364bf7abb2cbc2acf33fcfa35be0bf9da0" exitCode=6
Jul 16 23:01:17.065386 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.065261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerDied","Data":"9c0ff6fb123870b20687e3dc26a47c364bf7abb2cbc2acf33fcfa35be0bf9da0"}
Jul 16 23:01:17.065386 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.065332    2564 scope.go:117] "RemoveContainer" containerID="cbacab14e6ef2232093149594cee35c86295b125c21bcd3c46121cbc54b5e8f3"
Jul 16 23:01:17.065635 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.065607    2564 scope.go:117] "RemoveContainer" containerID="9c0ff6fb123870b20687e3dc26a47c364bf7abb2cbc2acf33fcfa35be0bf9da0"
Jul 16 23:01:17.065845 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:17.065823    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-mz7q8_odh-ai-gateway-infra(bc039722-05b8-47af-8e29-017147cc528d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" podUID="bc039722-05b8-47af-8e29-017147cc528d"
Jul 16 23:01:17.067689 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.067668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37dbe9bc-c669-4b91-837e-dba5faa19706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv\" (UID: \"37dbe9bc-c669-4b91-837e-dba5faa19706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:17.094365 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.094334    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"]
Jul 16 23:01:17.168709 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.168665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37dbe9bc-c669-4b91-837e-dba5faa19706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv\" (UID: \"37dbe9bc-c669-4b91-837e-dba5faa19706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:17.169336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.169315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37dbe9bc-c669-4b91-837e-dba5faa19706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv\" (UID: \"37dbe9bc-c669-4b91-837e-dba5faa19706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:17.318413 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.318323    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:17.454658 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.454628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"]
Jul 16 23:01:17.455993 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:01:17.455956    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37dbe9bc_c669_4b91_837e_dba5faa19706.slice/crio-66478c3d3b5abaaa07598789838609873ddafd791481beeb791e7e896fb2e160 WatchSource:0}: Error finding container 66478c3d3b5abaaa07598789838609873ddafd791481beeb791e7e896fb2e160: Status 404 returned error can't find the container with id 66478c3d3b5abaaa07598789838609873ddafd791481beeb791e7e896fb2e160
Jul 16 23:01:17.596991 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:17.596908    2564 scope.go:117] "RemoveContainer" containerID="63f151bf65ae2ac2f411c06bc2203ed5bb07b59f1d1279645422db3865cfe2ed"
Jul 16 23:01:18.075432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.075396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerStarted","Data":"4c4a1216abba6d9ad730faa3d9350f36ba3927f9e0dc0986bcd54ccc2f7dfbe5"}
Jul 16 23:01:18.075847 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.075455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerStarted","Data":"66478c3d3b5abaaa07598789838609873ddafd791481beeb791e7e896fb2e160"}
Jul 16 23:01:18.078833 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.078808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" event={"ID":"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7","Type":"ContainerStarted","Data":"843d3eb65a920446e9e0c1f34af0e466d5c87ecda52bc3e5bc0f3b5efa18ead7"}
Jul 16 23:01:18.093031 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.092973    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" podStartSLOduration=2.092953277 podStartE2EDuration="2.092953277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:01:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:01:18.091534012 +0000 UTC m=+2634.412090086" watchObservedRunningTime="2026-07-16 23:01:18.092953277 +0000 UTC m=+2634.413509350"
Jul 16 23:01:18.126527 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.126425    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" podStartSLOduration=16.126408754 podStartE2EDuration="16.126408754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:01:18.124506679 +0000 UTC m=+2634.445062751" watchObservedRunningTime="2026-07-16 23:01:18.126408754 +0000 UTC m=+2634.446964867"
Jul 16 23:01:18.240602 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.240571    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:01:18.276747 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.276719    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc039722-05b8-47af-8e29-017147cc528d-maas-api-service-ca\") pod \"bc039722-05b8-47af-8e29-017147cc528d\" (UID: \"bc039722-05b8-47af-8e29-017147cc528d\") "
Jul 16 23:01:18.277036 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.277016    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc039722-05b8-47af-8e29-017147cc528d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc039722-05b8-47af-8e29-017147cc528d" (UID: "bc039722-05b8-47af-8e29-017147cc528d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:01:18.377523 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:18.377430    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc039722-05b8-47af-8e29-017147cc528d-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:19.086153 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.086101    2564 generic.go:356] "Generic (PLEG): container finished" podID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerID="4c4a1216abba6d9ad730faa3d9350f36ba3927f9e0dc0986bcd54ccc2f7dfbe5" exitCode=6
Jul 16 23:01:19.086586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.086188    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerDied","Data":"4c4a1216abba6d9ad730faa3d9350f36ba3927f9e0dc0986bcd54ccc2f7dfbe5"}
Jul 16 23:01:19.086586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.086381    2564 scope.go:117] "RemoveContainer" containerID="4c4a1216abba6d9ad730faa3d9350f36ba3927f9e0dc0986bcd54ccc2f7dfbe5"
Jul 16 23:01:19.087688 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.087662    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"
Jul 16 23:01:19.087688 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.087679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8" event={"ID":"bc039722-05b8-47af-8e29-017147cc528d","Type":"ContainerDied","Data":"9e4f7fb390dcf5b0f7a05f65421ee6df31649923c5ea82f77eaf38635fec2e25"}
Jul 16 23:01:19.087839 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.087730    2564 scope.go:117] "RemoveContainer" containerID="9c0ff6fb123870b20687e3dc26a47c364bf7abb2cbc2acf33fcfa35be0bf9da0"
Jul 16 23:01:19.089682 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.089656    2564 generic.go:356] "Generic (PLEG): container finished" podID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerID="843d3eb65a920446e9e0c1f34af0e466d5c87ecda52bc3e5bc0f3b5efa18ead7" exitCode=6
Jul 16 23:01:19.089824 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.089718    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" event={"ID":"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7","Type":"ContainerDied","Data":"843d3eb65a920446e9e0c1f34af0e466d5c87ecda52bc3e5bc0f3b5efa18ead7"}
Jul 16 23:01:19.089993 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.089979    2564 scope.go:117] "RemoveContainer" containerID="843d3eb65a920446e9e0c1f34af0e466d5c87ecda52bc3e5bc0f3b5efa18ead7"
Jul 16 23:01:19.090241 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:19.090220    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq_odh-ai-gateway-infra(b855cee6-e63c-451e-b1b1-0eb9a63bd0c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7"
Jul 16 23:01:19.098882 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.098856    2564 scope.go:117] "RemoveContainer" containerID="63f151bf65ae2ac2f411c06bc2203ed5bb07b59f1d1279645422db3865cfe2ed"
Jul 16 23:01:19.120975 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.120947    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"]
Jul 16 23:01:19.125545 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.125524    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-mz7q8"]
Jul 16 23:01:19.137471 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.137446    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"]
Jul 16 23:01:19.602361 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:19.602327    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc039722-05b8-47af-8e29-017147cc528d" path="/var/lib/kubelet/pods/bc039722-05b8-47af-8e29-017147cc528d/volumes"
Jul 16 23:01:20.096348 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.096316    2564 generic.go:356] "Generic (PLEG): container finished" podID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerID="340624f6d6b16fd39651519e97c1cabd72102e884d1ef6093cfcc39b69cb8d64" exitCode=6
Jul 16 23:01:20.096835 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.096404    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerDied","Data":"340624f6d6b16fd39651519e97c1cabd72102e884d1ef6093cfcc39b69cb8d64"}
Jul 16 23:01:20.096835 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.096465    2564 scope.go:117] "RemoveContainer" containerID="4c4a1216abba6d9ad730faa3d9350f36ba3927f9e0dc0986bcd54ccc2f7dfbe5"
Jul 16 23:01:20.096835 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.096677    2564 scope.go:117] "RemoveContainer" containerID="340624f6d6b16fd39651519e97c1cabd72102e884d1ef6093cfcc39b69cb8d64"
Jul 16 23:01:20.096978 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:20.096894    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv_odh-ai-gateway-infra(37dbe9bc-c669-4b91-837e-dba5faa19706)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706"
Jul 16 23:01:20.232378 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.232347    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:20.296714 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.296676    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7-maas-api-service-ca\") pod \"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7\" (UID: \"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7\") "
Jul 16 23:01:20.297055 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.297031    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" (UID: "b855cee6-e63c-451e-b1b1-0eb9a63bd0c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:01:20.398290 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:20.398203    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:21.106853 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.106829    2564 scope.go:117] "RemoveContainer" containerID="340624f6d6b16fd39651519e97c1cabd72102e884d1ef6093cfcc39b69cb8d64"
Jul 16 23:01:21.107293 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:21.107040    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv_odh-ai-gateway-infra(37dbe9bc-c669-4b91-837e-dba5faa19706)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706"
Jul 16 23:01:21.107890 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.107872    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"
Jul 16 23:01:21.107997 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.107870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq" event={"ID":"b855cee6-e63c-451e-b1b1-0eb9a63bd0c7","Type":"ContainerDied","Data":"6d1e4e73acad2025f6785dc74f837135bcc22a9e7174f76004227e35ffd7f5a3"}
Jul 16 23:01:21.108037 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.108012    2564 scope.go:117] "RemoveContainer" containerID="843d3eb65a920446e9e0c1f34af0e466d5c87ecda52bc3e5bc0f3b5efa18ead7"
Jul 16 23:01:21.136378 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.136344    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"]
Jul 16 23:01:21.141401 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.141368    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-gd9gq"]
Jul 16 23:01:21.602198 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:21.602164    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" path="/var/lib/kubelet/pods/b855cee6-e63c-451e-b1b1-0eb9a63bd0c7/volumes"
Jul 16 23:01:34.077515 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:34.077481    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"]
Jul 16 23:01:34.078163 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:34.077712    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" containerID="cri-o://7ec436e3d86503e0ae106b6c9630f13bd2b907091bfe6c6906129842348f6be2" gracePeriod=30
Jul 16 23:01:35.033909 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:35.033870    2564 patch_prober.go:28] interesting pod/e2e-mt-25067845-openshift-default-69889c9f79-65d9c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 23:01:35.034077 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:35.033940    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 23:01:35.602926 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:35.602896    2564 scope.go:117] "RemoveContainer" containerID="340624f6d6b16fd39651519e97c1cabd72102e884d1ef6093cfcc39b69cb8d64"
Jul 16 23:01:36.198253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:36.198219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerStarted","Data":"5f3185275a9ab6ef0f05f9772d59244286d6b610906c37704a1fee14f676d845"}
Jul 16 23:01:37.205506 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:37.205474    2564 generic.go:356] "Generic (PLEG): container finished" podID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerID="5f3185275a9ab6ef0f05f9772d59244286d6b610906c37704a1fee14f676d845" exitCode=6
Jul 16 23:01:37.205951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:37.205515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerDied","Data":"5f3185275a9ab6ef0f05f9772d59244286d6b610906c37704a1fee14f676d845"}
Jul 16 23:01:37.205951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:37.205562    2564 scope.go:117] "RemoveContainer" containerID="340624f6d6b16fd39651519e97c1cabd72102e884d1ef6093cfcc39b69cb8d64"
Jul 16 23:01:37.205951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:37.205797    2564 scope.go:117] "RemoveContainer" containerID="5f3185275a9ab6ef0f05f9772d59244286d6b610906c37704a1fee14f676d845"
Jul 16 23:01:37.206172 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:37.206017    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv_odh-ai-gateway-infra(37dbe9bc-c669-4b91-837e-dba5faa19706)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706"
Jul 16 23:01:37.244670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:37.244637    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"]
Jul 16 23:01:37.889617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:37.889579    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:01:38.344239 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:38.344210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:38.461397 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:38.461363    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37dbe9bc-c669-4b91-837e-dba5faa19706-maas-api-service-ca\") pod \"37dbe9bc-c669-4b91-837e-dba5faa19706\" (UID: \"37dbe9bc-c669-4b91-837e-dba5faa19706\") "
Jul 16 23:01:38.461742 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:38.461720    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37dbe9bc-c669-4b91-837e-dba5faa19706-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37dbe9bc-c669-4b91-837e-dba5faa19706" (UID: "37dbe9bc-c669-4b91-837e-dba5faa19706"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:01:38.562347 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:38.562259    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37dbe9bc-c669-4b91-837e-dba5faa19706-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.222144 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.222109    2564 generic.go:356] "Generic (PLEG): container finished" podID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerID="7ec436e3d86503e0ae106b6c9630f13bd2b907091bfe6c6906129842348f6be2" exitCode=0
Jul 16 23:01:39.222322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.222186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" event={"ID":"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20","Type":"ContainerDied","Data":"7ec436e3d86503e0ae106b6c9630f13bd2b907091bfe6c6906129842348f6be2"}
Jul 16 23:01:39.222322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.222247    2564 scope.go:117] "RemoveContainer" containerID="4f359d49954fd7df5255c6c543cfd48fd4b97162a99566f9bbc1863ae14e8106"
Jul 16 23:01:39.223986 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.223965    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"
Jul 16 23:01:39.223986 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.223977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv" event={"ID":"37dbe9bc-c669-4b91-837e-dba5faa19706","Type":"ContainerDied","Data":"66478c3d3b5abaaa07598789838609873ddafd791481beeb791e7e896fb2e160"}
Jul 16 23:01:39.234272 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.234251    2564 scope.go:117] "RemoveContainer" containerID="5f3185275a9ab6ef0f05f9772d59244286d6b610906c37704a1fee14f676d845"
Jul 16 23:01:39.263005 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.262977    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"]
Jul 16 23:01:39.268571 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.268543    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-prwzv"]
Jul 16 23:01:39.335896 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.335865    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 23:01:39.468932 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.468897    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istiod-ca-cert\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.468995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpd9p\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-kube-api-access-cpd9p\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469023    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-credential-socket\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-podinfo\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469085    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-token\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469116    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-certs\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-envoy\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469213    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-socket\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469276    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-data\") pod \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\" (UID: \"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20\") "
Jul 16 23:01:39.469432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469330    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:01:39.469924 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469543    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:39.469924 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469562    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.469924 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469651    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:39.469924 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.469670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:39.470417 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.470389    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:39.471367 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.471337    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:01:39.471467 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.471411    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-kube-api-access-cpd9p" (OuterVolumeSpecName: "kube-api-access-cpd9p") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "kube-api-access-cpd9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:01:39.471467 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.471445    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:01:39.471562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.471469    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" (UID: "6b9f70d3-80b2-407c-98f8-3bf6f46b8c20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:39.570831 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570794    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpd9p\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-kube-api-access-cpd9p\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.570831 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570824    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.570831 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570833    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.570831 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570842    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.571100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570849    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.571100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570857    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.571100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570863    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.571100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.570870    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:01:39.601042 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:39.601008    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" path="/var/lib/kubelet/pods/37dbe9bc-c669-4b91-837e-dba5faa19706/volumes"
Jul 16 23:01:40.229990 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:40.229963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"
Jul 16 23:01:40.230216 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:40.229956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c" event={"ID":"6b9f70d3-80b2-407c-98f8-3bf6f46b8c20","Type":"ContainerDied","Data":"b6b40d24e40f2819afb65b854108133f3d5d8ea058967e953687cb2c978f3edc"}
Jul 16 23:01:40.230216 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:40.230073    2564 scope.go:117] "RemoveContainer" containerID="7ec436e3d86503e0ae106b6c9630f13bd2b907091bfe6c6906129842348f6be2"
Jul 16 23:01:40.252266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:40.252235    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"]
Jul 16 23:01:40.256856 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:40.256828    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-25067845-openshift-default-69889c9f79-65d9c"]
Jul 16 23:01:41.601921 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:41.601885    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" path="/var/lib/kubelet/pods/6b9f70d3-80b2-407c-98f8-3bf6f46b8c20/volumes"
Jul 16 23:01:48.912526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:48.912487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:01:50.481604 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.481566    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"]
Jul 16 23:01:50.482084 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482037    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482084 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482056    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482084 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482067    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482084 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482074    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482084 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482084    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482092    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482102    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482110    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482123    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482149    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482163    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482171    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482184    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482192    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482208    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482216    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482241    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482249    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482337    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482347    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482357    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482367    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482375    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482383    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482391    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482400    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9f70d3-80b2-407c-98f8-3bf6f46b8c20" containerName="istio-proxy"
Jul 16 23:01:50.482402 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482410    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:50.483595 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.482418    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:50.487353 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.487332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:01:50.492347 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.492325    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:01:50.492487 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.492354    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:01:50.501441 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.501417    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"]
Jul 16 23:01:50.568924 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.568877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b1ac94-514f-4364-8527-b3190b8d1d94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-nft76\" (UID: \"68b1ac94-514f-4364-8527-b3190b8d1d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:01:50.670116 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.670076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b1ac94-514f-4364-8527-b3190b8d1d94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-nft76\" (UID: \"68b1ac94-514f-4364-8527-b3190b8d1d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:01:50.670696 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.670675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b1ac94-514f-4364-8527-b3190b8d1d94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-nft76\" (UID: \"68b1ac94-514f-4364-8527-b3190b8d1d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:01:50.797713 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.797618    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:01:50.976551 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:50.976528    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"]
Jul 16 23:01:50.978482 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:01:50.978459    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b1ac94_514f_4364_8527_b3190b8d1d94.slice/crio-c86b3967685c0e11466cdde9558ffc497496167085f4016acc24e603df26c644 WatchSource:0}: Error finding container c86b3967685c0e11466cdde9558ffc497496167085f4016acc24e603df26c644: Status 404 returned error can't find the container with id c86b3967685c0e11466cdde9558ffc497496167085f4016acc24e603df26c644
Jul 16 23:01:51.295193 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:51.295162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" event={"ID":"68b1ac94-514f-4364-8527-b3190b8d1d94","Type":"ContainerStarted","Data":"9eff88bd7d3bbc8fcfc2cab0e1722793f20904534bfe666bd330d52f96df81cb"}
Jul 16 23:01:51.295336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:51.295218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" event={"ID":"68b1ac94-514f-4364-8527-b3190b8d1d94","Type":"ContainerStarted","Data":"c86b3967685c0e11466cdde9558ffc497496167085f4016acc24e603df26c644"}
Jul 16 23:01:51.311443 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:51.311392    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" podStartSLOduration=1.311376133 podStartE2EDuration="1.311376133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:01:51.310375707 +0000 UTC m=+2667.630931779" watchObservedRunningTime="2026-07-16 23:01:51.311376133 +0000 UTC m=+2667.631932204"
Jul 16 23:01:52.301716 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.301681    2564 generic.go:356] "Generic (PLEG): container finished" podID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerID="9eff88bd7d3bbc8fcfc2cab0e1722793f20904534bfe666bd330d52f96df81cb" exitCode=6
Jul 16 23:01:52.302130 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.301779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" event={"ID":"68b1ac94-514f-4364-8527-b3190b8d1d94","Type":"ContainerDied","Data":"9eff88bd7d3bbc8fcfc2cab0e1722793f20904534bfe666bd330d52f96df81cb"}
Jul 16 23:01:52.302130 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.301972    2564 scope.go:117] "RemoveContainer" containerID="9eff88bd7d3bbc8fcfc2cab0e1722793f20904534bfe666bd330d52f96df81cb"
Jul 16 23:01:52.375311 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.375282    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"]
Jul 16 23:01:52.375626 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.375614    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:52.375626 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.375627    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc039722-05b8-47af-8e29-017147cc528d" containerName="revoke-keys"
Jul 16 23:01:52.375714 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.375641    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:52.375714 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.375646    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b855cee6-e63c-451e-b1b1-0eb9a63bd0c7" containerName="revoke-keys"
Jul 16 23:01:52.375714 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.375695    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37dbe9bc-c669-4b91-837e-dba5faa19706" containerName="revoke-keys"
Jul 16 23:01:52.379799 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.379782    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:01:52.385872 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.385843    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"]
Jul 16 23:01:52.485237 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.485210    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3151d1a8-0b48-4b8f-9b56-a8a38603ead3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld\" (UID: \"3151d1a8-0b48-4b8f-9b56-a8a38603ead3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:01:52.586368 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.586331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3151d1a8-0b48-4b8f-9b56-a8a38603ead3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld\" (UID: \"3151d1a8-0b48-4b8f-9b56-a8a38603ead3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:01:52.586945 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.586922    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3151d1a8-0b48-4b8f-9b56-a8a38603ead3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld\" (UID: \"3151d1a8-0b48-4b8f-9b56-a8a38603ead3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:01:52.692349 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:52.692303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:01:53.036704 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:53.036676    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"]
Jul 16 23:01:53.037906 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:01:53.037881    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3151d1a8_0b48_4b8f_9b56_a8a38603ead3.slice/crio-ee161f21ad4c1d7724f98541616f97d8d790e2fce9d85d61fc6f79148a05fc6a WatchSource:0}: Error finding container ee161f21ad4c1d7724f98541616f97d8d790e2fce9d85d61fc6f79148a05fc6a: Status 404 returned error can't find the container with id ee161f21ad4c1d7724f98541616f97d8d790e2fce9d85d61fc6f79148a05fc6a
Jul 16 23:01:53.308243 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:53.308211    2564 generic.go:356] "Generic (PLEG): container finished" podID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerID="e1329f665ae9b0a42bfbd70fe760233926b5909abeca874bf809778f9c6965e4" exitCode=6
Jul 16 23:01:53.308651 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:53.308270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" event={"ID":"68b1ac94-514f-4364-8527-b3190b8d1d94","Type":"ContainerDied","Data":"e1329f665ae9b0a42bfbd70fe760233926b5909abeca874bf809778f9c6965e4"}
Jul 16 23:01:53.308651 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:53.308336    2564 scope.go:117] "RemoveContainer" containerID="9eff88bd7d3bbc8fcfc2cab0e1722793f20904534bfe666bd330d52f96df81cb"
Jul 16 23:01:53.308651 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:53.308483    2564 scope.go:117] "RemoveContainer" containerID="e1329f665ae9b0a42bfbd70fe760233926b5909abeca874bf809778f9c6965e4"
Jul 16 23:01:53.308812 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:53.308731    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-nft76_odh-ai-gateway-infra(68b1ac94-514f-4364-8527-b3190b8d1d94)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94"
Jul 16 23:01:53.309879 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:53.309854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" event={"ID":"3151d1a8-0b48-4b8f-9b56-a8a38603ead3","Type":"ContainerStarted","Data":"ee161f21ad4c1d7724f98541616f97d8d790e2fce9d85d61fc6f79148a05fc6a"}
Jul 16 23:01:54.317251 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:54.317223    2564 scope.go:117] "RemoveContainer" containerID="e1329f665ae9b0a42bfbd70fe760233926b5909abeca874bf809778f9c6965e4"
Jul 16 23:01:54.317692 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:54.317417    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-nft76_odh-ai-gateway-infra(68b1ac94-514f-4364-8527-b3190b8d1d94)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94"
Jul 16 23:01:54.318602 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:54.318578    2564 generic.go:356] "Generic (PLEG): container finished" podID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerID="6c3671ad00794c25759146e58ff4269208131b23ab20be06de42a99ec9c258fd" exitCode=6
Jul 16 23:01:54.318703 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:54.318636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" event={"ID":"3151d1a8-0b48-4b8f-9b56-a8a38603ead3","Type":"ContainerDied","Data":"6c3671ad00794c25759146e58ff4269208131b23ab20be06de42a99ec9c258fd"}
Jul 16 23:01:54.318803 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:54.318792    2564 scope.go:117] "RemoveContainer" containerID="6c3671ad00794c25759146e58ff4269208131b23ab20be06de42a99ec9c258fd"
Jul 16 23:01:55.325906 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:55.325872    2564 generic.go:356] "Generic (PLEG): container finished" podID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerID="ea4a7101da8d2efed665511d89ff6113599d4c691a8c378c78bf60a9922116c7" exitCode=6
Jul 16 23:01:55.326416 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:55.325958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" event={"ID":"3151d1a8-0b48-4b8f-9b56-a8a38603ead3","Type":"ContainerDied","Data":"ea4a7101da8d2efed665511d89ff6113599d4c691a8c378c78bf60a9922116c7"}
Jul 16 23:01:55.326416 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:55.326031    2564 scope.go:117] "RemoveContainer" containerID="6c3671ad00794c25759146e58ff4269208131b23ab20be06de42a99ec9c258fd"
Jul 16 23:01:55.326416 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:55.326269    2564 scope.go:117] "RemoveContainer" containerID="ea4a7101da8d2efed665511d89ff6113599d4c691a8c378c78bf60a9922116c7"
Jul 16 23:01:55.326607 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:55.326451    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld_odh-ai-gateway-infra(3151d1a8-0b48-4b8f-9b56-a8a38603ead3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3"
Jul 16 23:01:56.332872 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:56.332832    2564 scope.go:117] "RemoveContainer" containerID="ea4a7101da8d2efed665511d89ff6113599d4c691a8c378c78bf60a9922116c7"
Jul 16 23:01:56.333248 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:01:56.333012    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld_odh-ai-gateway-infra(3151d1a8-0b48-4b8f-9b56-a8a38603ead3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3"
Jul 16 23:01:58.917825 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:58.917786    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"]
Jul 16 23:01:58.922528 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:58.922510    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:01:58.929635 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:58.929608    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"]
Jul 16 23:01:59.038075 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:59.038033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d08bb45-7b8c-4773-8e6a-b842ef914d57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-5qsf6\" (UID: \"2d08bb45-7b8c-4773-8e6a-b842ef914d57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:01:59.139312 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:59.139275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d08bb45-7b8c-4773-8e6a-b842ef914d57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-5qsf6\" (UID: \"2d08bb45-7b8c-4773-8e6a-b842ef914d57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:01:59.139884 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:59.139865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d08bb45-7b8c-4773-8e6a-b842ef914d57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-5qsf6\" (UID: \"2d08bb45-7b8c-4773-8e6a-b842ef914d57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:01:59.233817 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:59.233773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:01:59.370360 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:01:59.370334    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"]
Jul 16 23:01:59.372059 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:01:59.372029    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d08bb45_7b8c_4773_8e6a_b842ef914d57.slice/crio-7767e065ae75ad55fe62219c0d58e15c26936d56a2ace356c75ee1674593e6b8 WatchSource:0}: Error finding container 7767e065ae75ad55fe62219c0d58e15c26936d56a2ace356c75ee1674593e6b8: Status 404 returned error can't find the container with id 7767e065ae75ad55fe62219c0d58e15c26936d56a2ace356c75ee1674593e6b8
Jul 16 23:02:00.356819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:00.356728    2564 generic.go:356] "Generic (PLEG): container finished" podID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerID="90cbbd5912ad1735b659bd36ee6a2bca5c21c95def7a56f05177bdaf87aab0b7" exitCode=6
Jul 16 23:02:00.356819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:00.356808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" event={"ID":"2d08bb45-7b8c-4773-8e6a-b842ef914d57","Type":"ContainerDied","Data":"90cbbd5912ad1735b659bd36ee6a2bca5c21c95def7a56f05177bdaf87aab0b7"}
Jul 16 23:02:00.357240 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:00.356859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" event={"ID":"2d08bb45-7b8c-4773-8e6a-b842ef914d57","Type":"ContainerStarted","Data":"7767e065ae75ad55fe62219c0d58e15c26936d56a2ace356c75ee1674593e6b8"}
Jul 16 23:02:00.357240 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:00.357018    2564 scope.go:117] "RemoveContainer" containerID="90cbbd5912ad1735b659bd36ee6a2bca5c21c95def7a56f05177bdaf87aab0b7"
Jul 16 23:02:01.363570 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:01.363484    2564 generic.go:356] "Generic (PLEG): container finished" podID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerID="c61fe304d52bf73312891fcc4fdcc5529400f60eb1178d4748b4f36b0efce6f2" exitCode=6
Jul 16 23:02:01.363979 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:01.363570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" event={"ID":"2d08bb45-7b8c-4773-8e6a-b842ef914d57","Type":"ContainerDied","Data":"c61fe304d52bf73312891fcc4fdcc5529400f60eb1178d4748b4f36b0efce6f2"}
Jul 16 23:02:01.363979 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:01.363624    2564 scope.go:117] "RemoveContainer" containerID="90cbbd5912ad1735b659bd36ee6a2bca5c21c95def7a56f05177bdaf87aab0b7"
Jul 16 23:02:01.363979 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:01.363763    2564 scope.go:117] "RemoveContainer" containerID="c61fe304d52bf73312891fcc4fdcc5529400f60eb1178d4748b4f36b0efce6f2"
Jul 16 23:02:01.363979 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:01.363968    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-5qsf6_odh-ai-gateway-infra(2d08bb45-7b8c-4773-8e6a-b842ef914d57)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57"
Jul 16 23:02:02.370371 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:02.370340    2564 scope.go:117] "RemoveContainer" containerID="c61fe304d52bf73312891fcc4fdcc5529400f60eb1178d4748b4f36b0efce6f2"
Jul 16 23:02:02.370761 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:02.370560    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-5qsf6_odh-ai-gateway-infra(2d08bb45-7b8c-4773-8e6a-b842ef914d57)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57"
Jul 16 23:02:06.596728 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:06.596696    2564 scope.go:117] "RemoveContainer" containerID="e1329f665ae9b0a42bfbd70fe760233926b5909abeca874bf809778f9c6965e4"
Jul 16 23:02:07.398304 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:07.398270    2564 generic.go:356] "Generic (PLEG): container finished" podID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerID="0b24f95e2b25c03abaf03572c19b766b3685efe58c247f9d8eff679db730c695" exitCode=6
Jul 16 23:02:07.398485 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:07.398329    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" event={"ID":"68b1ac94-514f-4364-8527-b3190b8d1d94","Type":"ContainerDied","Data":"0b24f95e2b25c03abaf03572c19b766b3685efe58c247f9d8eff679db730c695"}
Jul 16 23:02:07.398485 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:07.398384    2564 scope.go:117] "RemoveContainer" containerID="e1329f665ae9b0a42bfbd70fe760233926b5909abeca874bf809778f9c6965e4"
Jul 16 23:02:07.398718 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:07.398697    2564 scope.go:117] "RemoveContainer" containerID="0b24f95e2b25c03abaf03572c19b766b3685efe58c247f9d8eff679db730c695"
Jul 16 23:02:07.398979 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:07.398956    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-nft76_odh-ai-gateway-infra(68b1ac94-514f-4364-8527-b3190b8d1d94)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94"
Jul 16 23:02:07.602200 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:07.602168    2564 scope.go:117] "RemoveContainer" containerID="ea4a7101da8d2efed665511d89ff6113599d4c691a8c378c78bf60a9922116c7"
Jul 16 23:02:08.405850 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.405813    2564 generic.go:356] "Generic (PLEG): container finished" podID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerID="3f899606200f4ae319b3a2fb4808f991d79eddc86ad5b2eb33b513515e0e3164" exitCode=6
Jul 16 23:02:08.406053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.405890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" event={"ID":"3151d1a8-0b48-4b8f-9b56-a8a38603ead3","Type":"ContainerDied","Data":"3f899606200f4ae319b3a2fb4808f991d79eddc86ad5b2eb33b513515e0e3164"}
Jul 16 23:02:08.406053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.405986    2564 scope.go:117] "RemoveContainer" containerID="ea4a7101da8d2efed665511d89ff6113599d4c691a8c378c78bf60a9922116c7"
Jul 16 23:02:08.406359 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.406338    2564 scope.go:117] "RemoveContainer" containerID="3f899606200f4ae319b3a2fb4808f991d79eddc86ad5b2eb33b513515e0e3164"
Jul 16 23:02:08.406597 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:08.406575    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld_odh-ai-gateway-infra(3151d1a8-0b48-4b8f-9b56-a8a38603ead3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3"
Jul 16 23:02:08.422911 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.422877    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"]
Jul 16 23:02:08.552257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.552231    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:02:08.628026 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.627985    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b1ac94-514f-4364-8527-b3190b8d1d94-maas-api-service-ca\") pod \"68b1ac94-514f-4364-8527-b3190b8d1d94\" (UID: \"68b1ac94-514f-4364-8527-b3190b8d1d94\") "
Jul 16 23:02:08.628419 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.628354    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b1ac94-514f-4364-8527-b3190b8d1d94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68b1ac94-514f-4364-8527-b3190b8d1d94" (UID: "68b1ac94-514f-4364-8527-b3190b8d1d94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:02:08.728847 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:08.728809    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68b1ac94-514f-4364-8527-b3190b8d1d94-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:02:09.414613 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.414583    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"
Jul 16 23:02:09.414795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.414627    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76" event={"ID":"68b1ac94-514f-4364-8527-b3190b8d1d94","Type":"ContainerDied","Data":"c86b3967685c0e11466cdde9558ffc497496167085f4016acc24e603df26c644"}
Jul 16 23:02:09.414795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.414683    2564 scope.go:117] "RemoveContainer" containerID="0b24f95e2b25c03abaf03572c19b766b3685efe58c247f9d8eff679db730c695"
Jul 16 23:02:09.436104 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.436063    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"]
Jul 16 23:02:09.441966 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.441939    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"]
Jul 16 23:02:09.447956 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.447929    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-nft76"]
Jul 16 23:02:09.570631 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.570605    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:02:09.601543 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.601514    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" path="/var/lib/kubelet/pods/68b1ac94-514f-4364-8527-b3190b8d1d94/volumes"
Jul 16 23:02:09.635900 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.635872    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3151d1a8-0b48-4b8f-9b56-a8a38603ead3-maas-api-service-ca\") pod \"3151d1a8-0b48-4b8f-9b56-a8a38603ead3\" (UID: \"3151d1a8-0b48-4b8f-9b56-a8a38603ead3\") "
Jul 16 23:02:09.636382 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.636211    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3151d1a8-0b48-4b8f-9b56-a8a38603ead3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3151d1a8-0b48-4b8f-9b56-a8a38603ead3" (UID: "3151d1a8-0b48-4b8f-9b56-a8a38603ead3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:02:09.737453 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:09.737420    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3151d1a8-0b48-4b8f-9b56-a8a38603ead3-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:02:10.423378 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:10.423347    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"
Jul 16 23:02:10.423565 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:10.423395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld" event={"ID":"3151d1a8-0b48-4b8f-9b56-a8a38603ead3","Type":"ContainerDied","Data":"ee161f21ad4c1d7724f98541616f97d8d790e2fce9d85d61fc6f79148a05fc6a"}
Jul 16 23:02:10.423565 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:10.423458    2564 scope.go:117] "RemoveContainer" containerID="3f899606200f4ae319b3a2fb4808f991d79eddc86ad5b2eb33b513515e0e3164"
Jul 16 23:02:10.446872 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:10.446846    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"]
Jul 16 23:02:10.454338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:10.454311    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-7hrld"]
Jul 16 23:02:11.602336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:11.602300    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" path="/var/lib/kubelet/pods/3151d1a8-0b48-4b8f-9b56-a8a38603ead3/volumes"
Jul 16 23:02:17.067678 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.067597    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"]
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.067971    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.067984    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.067993    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.067999    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068012    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068018    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068033    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068038    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068045    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068050    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068102    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:17.068388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068110    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068117    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068123    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3151d1a8-0b48-4b8f-9b56-a8a38603ead3" containerName="revoke-keys"
Jul 16 23:02:17.068388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.068129    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:17.071173 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.071156    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:17.078610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.078583    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"]
Jul 16 23:02:17.103782 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.103748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f857a59-5b45-416f-a87a-8b2b70442b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj\" (UID: \"3f857a59-5b45-416f-a87a-8b2b70442b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:17.204164 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.204117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f857a59-5b45-416f-a87a-8b2b70442b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj\" (UID: \"3f857a59-5b45-416f-a87a-8b2b70442b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:17.204781 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.204754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f857a59-5b45-416f-a87a-8b2b70442b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj\" (UID: \"3f857a59-5b45-416f-a87a-8b2b70442b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:17.382286 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.382198    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:17.512644 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.512622    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"]
Jul 16 23:02:17.514100 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:02:17.514069    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f857a59_5b45_416f_a87a_8b2b70442b6e.slice/crio-8053e2b202fa00550c9d275285c7c990829164dce7f7f7062bb7e749c9a1c486 WatchSource:0}: Error finding container 8053e2b202fa00550c9d275285c7c990829164dce7f7f7062bb7e749c9a1c486: Status 404 returned error can't find the container with id 8053e2b202fa00550c9d275285c7c990829164dce7f7f7062bb7e749c9a1c486
Jul 16 23:02:17.596948 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:17.596916    2564 scope.go:117] "RemoveContainer" containerID="c61fe304d52bf73312891fcc4fdcc5529400f60eb1178d4748b4f36b0efce6f2"
Jul 16 23:02:18.477219 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.477186    2564 generic.go:356] "Generic (PLEG): container finished" podID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerID="79883945569f3c2c4cc72cf63ea491090203296f8d8a7166cebc0562df0f9315" exitCode=6
Jul 16 23:02:18.477706 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.477255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" event={"ID":"2d08bb45-7b8c-4773-8e6a-b842ef914d57","Type":"ContainerDied","Data":"79883945569f3c2c4cc72cf63ea491090203296f8d8a7166cebc0562df0f9315"}
Jul 16 23:02:18.477706 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.477312    2564 scope.go:117] "RemoveContainer" containerID="c61fe304d52bf73312891fcc4fdcc5529400f60eb1178d4748b4f36b0efce6f2"
Jul 16 23:02:18.477706 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.477623    2564 scope.go:117] "RemoveContainer" containerID="79883945569f3c2c4cc72cf63ea491090203296f8d8a7166cebc0562df0f9315"
Jul 16 23:02:18.477894 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:18.477851    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-5qsf6_odh-ai-gateway-infra(2d08bb45-7b8c-4773-8e6a-b842ef914d57)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57"
Jul 16 23:02:18.478916 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.478895    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerID="fa7bc4514059735c7f1754e21df54d215cf36500497320b2218330c87fb8e652" exitCode=6
Jul 16 23:02:18.479076 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.479011    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" event={"ID":"3f857a59-5b45-416f-a87a-8b2b70442b6e","Type":"ContainerDied","Data":"fa7bc4514059735c7f1754e21df54d215cf36500497320b2218330c87fb8e652"}
Jul 16 23:02:18.479076 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.479055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" event={"ID":"3f857a59-5b45-416f-a87a-8b2b70442b6e","Type":"ContainerStarted","Data":"8053e2b202fa00550c9d275285c7c990829164dce7f7f7062bb7e749c9a1c486"}
Jul 16 23:02:18.479193 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:18.479094    2564 scope.go:117] "RemoveContainer" containerID="fa7bc4514059735c7f1754e21df54d215cf36500497320b2218330c87fb8e652"
Jul 16 23:02:19.487873 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.487838    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerID="d50948ca12aa0e5401124b188f51bd5f09d53a1e22d9bb1565e963a0f4236a37" exitCode=6
Jul 16 23:02:19.488357 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.487895    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" event={"ID":"3f857a59-5b45-416f-a87a-8b2b70442b6e","Type":"ContainerDied","Data":"d50948ca12aa0e5401124b188f51bd5f09d53a1e22d9bb1565e963a0f4236a37"}
Jul 16 23:02:19.488357 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.487995    2564 scope.go:117] "RemoveContainer" containerID="fa7bc4514059735c7f1754e21df54d215cf36500497320b2218330c87fb8e652"
Jul 16 23:02:19.488357 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.488165    2564 scope.go:117] "RemoveContainer" containerID="d50948ca12aa0e5401124b188f51bd5f09d53a1e22d9bb1565e963a0f4236a37"
Jul 16 23:02:19.488473 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:19.488429    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj_odh-ai-gateway-infra(3f857a59-5b45-416f-a87a-8b2b70442b6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e"
Jul 16 23:02:19.510189 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.510162    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"]
Jul 16 23:02:19.640933 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.640909    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:02:19.726311 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.726278    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d08bb45-7b8c-4773-8e6a-b842ef914d57-maas-api-service-ca\") pod \"2d08bb45-7b8c-4773-8e6a-b842ef914d57\" (UID: \"2d08bb45-7b8c-4773-8e6a-b842ef914d57\") "
Jul 16 23:02:19.726657 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.726634    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d08bb45-7b8c-4773-8e6a-b842ef914d57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d08bb45-7b8c-4773-8e6a-b842ef914d57" (UID: "2d08bb45-7b8c-4773-8e6a-b842ef914d57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:02:19.827659 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:19.827569    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d08bb45-7b8c-4773-8e6a-b842ef914d57-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:02:20.495287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:20.495258    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"
Jul 16 23:02:20.495731 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:20.495301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6" event={"ID":"2d08bb45-7b8c-4773-8e6a-b842ef914d57","Type":"ContainerDied","Data":"7767e065ae75ad55fe62219c0d58e15c26936d56a2ace356c75ee1674593e6b8"}
Jul 16 23:02:20.495731 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:20.495370    2564 scope.go:117] "RemoveContainer" containerID="79883945569f3c2c4cc72cf63ea491090203296f8d8a7166cebc0562df0f9315"
Jul 16 23:02:20.500331 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:20.500302    2564 scope.go:117] "RemoveContainer" containerID="d50948ca12aa0e5401124b188f51bd5f09d53a1e22d9bb1565e963a0f4236a37"
Jul 16 23:02:20.500531 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:20.500510    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj_odh-ai-gateway-infra(3f857a59-5b45-416f-a87a-8b2b70442b6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e"
Jul 16 23:02:20.533344 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:20.533313    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"]
Jul 16 23:02:20.537285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:20.537259    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-5qsf6"]
Jul 16 23:02:21.601817 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:21.601782    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" path="/var/lib/kubelet/pods/2d08bb45-7b8c-4773-8e6a-b842ef914d57/volumes"
Jul 16 23:02:24.580740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:24.580712    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 23:02:24.591953 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:24.591933    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 23:02:28.933548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.933510    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"]
Jul 16 23:02:28.934053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934038    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.934111 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934057    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.934111 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934078    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.934111 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934087    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.934111 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934104    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:28.934257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934113    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:28.934257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934205    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b1ac94-514f-4364-8527-b3190b8d1d94" containerName="revoke-keys"
Jul 16 23:02:28.934257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934216    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.934257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934227    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.934257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.934237    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:28.939057 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.939038    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:28.941315 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.941297    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bf84b0-openshift-default-dockercfg-bktst\""
Jul 16 23:02:28.949195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:28.949169    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"]
Jul 16 23:02:29.012801 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-envoy\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.012977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-certs\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.012977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5txp\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-kube-api-access-r5txp\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.012977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f494fee-5fb7-45e4-94dd-e2343411ca37-istiod-ca-cert\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.012977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-podinfo\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.012977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-data\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.012977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.012972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-socket\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.013202 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.013046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-credential-socket\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.013202 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.013085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-token\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.113616 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-envoy\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.113819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-certs\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.113819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5txp\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-kube-api-access-r5txp\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.113819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f494fee-5fb7-45e4-94dd-e2343411ca37-istiod-ca-cert\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.113819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-podinfo\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114061 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-data\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114061 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-socket\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114061 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.113974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-credential-socket\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114061 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.114024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-certs\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114061 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.114026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-token\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114351 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.114228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-data\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114410 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.114383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-socket\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114456 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.114438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-credential-socket\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.114502 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.114482    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f494fee-5fb7-45e4-94dd-e2343411ca37-istiod-ca-cert\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.116023 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.116003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-envoy\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.116228 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.116210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-podinfo\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.123638 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.123614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-token\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.124162 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.124095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5txp\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-kube-api-access-r5txp\") pod \"e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") " pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.250640 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.250607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:29.397967 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.396084    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"]
Jul 16 23:02:29.398815 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:02:29.398786    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f494fee_5fb7_45e4_94dd_e2343411ca37.slice/crio-03fcfec1716cf72ae4531af78b76d83216565940b6b687b0c2f324d9ae39315a WatchSource:0}: Error finding container 03fcfec1716cf72ae4531af78b76d83216565940b6b687b0c2f324d9ae39315a: Status 404 returned error can't find the container with id 03fcfec1716cf72ae4531af78b76d83216565940b6b687b0c2f324d9ae39315a
Jul 16 23:02:29.401049 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.401012    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:29.401178 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.401097    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:29.401252 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.401169    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:29.550715 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.550620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" event={"ID":"5f494fee-5fb7-45e4-94dd-e2343411ca37","Type":"ContainerStarted","Data":"e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"}
Jul 16 23:02:29.550715 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.550672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" event={"ID":"5f494fee-5fb7-45e4-94dd-e2343411ca37","Type":"ContainerStarted","Data":"03fcfec1716cf72ae4531af78b76d83216565940b6b687b0c2f324d9ae39315a"}
Jul 16 23:02:29.570590 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:29.570534    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podStartSLOduration=1.57051131 podStartE2EDuration="1.57051131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:02:29.569706285 +0000 UTC m=+2705.890262358" watchObservedRunningTime="2026-07-16 23:02:29.57051131 +0000 UTC m=+2705.891067382"
Jul 16 23:02:30.251305 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:30.251267    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:30.256381 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:30.256358    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:30.555829 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:30.555748    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:30.556608 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:30.556590    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:34.596834 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:34.596801    2564 scope.go:117] "RemoveContainer" containerID="d50948ca12aa0e5401124b188f51bd5f09d53a1e22d9bb1565e963a0f4236a37"
Jul 16 23:02:35.584610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:35.584579    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerID="2ba89609492a2dd20d60618caf4821b4e5b483664b3d6b8836f4672d5c3da80a" exitCode=6
Jul 16 23:02:35.584804 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:35.584632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" event={"ID":"3f857a59-5b45-416f-a87a-8b2b70442b6e","Type":"ContainerDied","Data":"2ba89609492a2dd20d60618caf4821b4e5b483664b3d6b8836f4672d5c3da80a"}
Jul 16 23:02:35.584804 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:35.584692    2564 scope.go:117] "RemoveContainer" containerID="d50948ca12aa0e5401124b188f51bd5f09d53a1e22d9bb1565e963a0f4236a37"
Jul 16 23:02:35.584999 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:35.584974    2564 scope.go:117] "RemoveContainer" containerID="2ba89609492a2dd20d60618caf4821b4e5b483664b3d6b8836f4672d5c3da80a"
Jul 16 23:02:35.585268 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:35.585247    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj_odh-ai-gateway-infra(3f857a59-5b45-416f-a87a-8b2b70442b6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e"
Jul 16 23:02:36.518153 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.518110    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"]
Jul 16 23:02:36.518638 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.518483    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:36.518638 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.518500    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d08bb45-7b8c-4773-8e6a-b842ef914d57" containerName="revoke-keys"
Jul 16 23:02:36.522912 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.522890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.525022 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.524998    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-30cb29-openshift-default-dockercfg-q77wd\""
Jul 16 23:02:36.534471 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.534446    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"]
Jul 16 23:02:36.620502 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.620464    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"]
Jul 16 23:02:36.689920 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.689876    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-podinfo\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690090 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.689923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-token\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690090 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690017    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cfefab-0cb2-488b-af0e-def2b67261b5-istiod-ca-cert\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690289 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-certs\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690289 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-socket\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690289 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq6ck\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-kube-api-access-sq6ck\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690289 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-envoy\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690486 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-data\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.690486 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.690340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-credential-socket\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.761667 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.761640    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:36.791786 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.791696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-credential-socket\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.791930 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.791783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-podinfo\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.791930 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.791814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-token\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.791949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cfefab-0cb2-488b-af0e-def2b67261b5-istiod-ca-cert\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-credential-socket\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792206 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-certs\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792206 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-socket\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792206 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq6ck\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-kube-api-access-sq6ck\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792369 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792202    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-envoy\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792369 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-data\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792369 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-certs\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-data\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cfefab-0cb2-488b-af0e-def2b67261b5-istiod-ca-cert\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.792740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.792736    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-socket\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.794378 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.794342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-envoy\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.794573 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.794549    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-podinfo\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.799758 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.799725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-token\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.799874 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.799836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq6ck\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-kube-api-access-sq6ck\") pod \"e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") " pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.834928 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.834879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:36.893990 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.893687    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f857a59-5b45-416f-a87a-8b2b70442b6e-maas-api-service-ca\") pod \"3f857a59-5b45-416f-a87a-8b2b70442b6e\" (UID: \"3f857a59-5b45-416f-a87a-8b2b70442b6e\") "
Jul 16 23:02:36.894174 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.894098    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f857a59-5b45-416f-a87a-8b2b70442b6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f857a59-5b45-416f-a87a-8b2b70442b6e" (UID: "3f857a59-5b45-416f-a87a-8b2b70442b6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:02:36.976159 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.976112    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"]
Jul 16 23:02:36.976694 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:02:36.976672    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9cfefab_0cb2_488b_af0e_def2b67261b5.slice/crio-88d82edfd1e2fe3114660e9fb3fbf08665ed330dc0baeb8537614b67e6685b66 WatchSource:0}: Error finding container 88d82edfd1e2fe3114660e9fb3fbf08665ed330dc0baeb8537614b67e6685b66: Status 404 returned error can't find the container with id 88d82edfd1e2fe3114660e9fb3fbf08665ed330dc0baeb8537614b67e6685b66
Jul 16 23:02:36.979268 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.979238    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:36.979335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.979309    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:36.979372 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.979338    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:36.995075 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:36.995049    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f857a59-5b45-416f-a87a-8b2b70442b6e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:02:37.606757 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.606717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" event={"ID":"a9cfefab-0cb2-488b-af0e-def2b67261b5","Type":"ContainerStarted","Data":"39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"}
Jul 16 23:02:37.607267 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.606785    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" event={"ID":"a9cfefab-0cb2-488b-af0e-def2b67261b5","Type":"ContainerStarted","Data":"88d82edfd1e2fe3114660e9fb3fbf08665ed330dc0baeb8537614b67e6685b66"}
Jul 16 23:02:37.607267 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.606809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj" event={"ID":"3f857a59-5b45-416f-a87a-8b2b70442b6e","Type":"ContainerDied","Data":"8053e2b202fa00550c9d275285c7c990829164dce7f7f7062bb7e749c9a1c486"}
Jul 16 23:02:37.607267 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.606826    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"
Jul 16 23:02:37.607267 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.606841    2564 scope.go:117] "RemoveContainer" containerID="2ba89609492a2dd20d60618caf4821b4e5b483664b3d6b8836f4672d5c3da80a"
Jul 16 23:02:37.625547 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.625492    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podStartSLOduration=1.625475472 podStartE2EDuration="1.625475472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:02:37.62433748 +0000 UTC m=+2713.944893551" watchObservedRunningTime="2026-07-16 23:02:37.625475472 +0000 UTC m=+2713.946031594"
Jul 16 23:02:37.641311 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.641278    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"]
Jul 16 23:02:37.647050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.647023    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-nfglj"]
Jul 16 23:02:37.836013 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:37.835972    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:38.580688 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:38.580651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:02:38.840340 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:38.840261    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:39.602436 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:39.602398    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" path="/var/lib/kubelet/pods/3f857a59-5b45-416f-a87a-8b2b70442b6e/volumes"
Jul 16 23:02:39.618279 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:39.618250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:39.619418 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:39.619400    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:42.417163 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417117    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"]
Jul 16 23:02:42.417553 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417516    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.417553 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417528    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.417553 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417550    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.417553 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417555    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.417705 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417626    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.417705 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417635    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.417705 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.417644    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:02:42.421951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.421933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:02:42.424441 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.424412    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:02:42.425398 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.425374    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:02:42.427990 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.427962    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"]
Jul 16 23:02:42.546895 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.546856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85416faf-a9e8-4d36-8187-35b65fa07f4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt\" (UID: \"85416faf-a9e8-4d36-8187-35b65fa07f4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:02:42.647762 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.647724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85416faf-a9e8-4d36-8187-35b65fa07f4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt\" (UID: \"85416faf-a9e8-4d36-8187-35b65fa07f4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:02:42.648384 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.648363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85416faf-a9e8-4d36-8187-35b65fa07f4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt\" (UID: \"85416faf-a9e8-4d36-8187-35b65fa07f4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:02:42.734884 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.734848    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:02:42.919360 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:42.919333    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"]
Jul 16 23:02:42.920780 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:02:42.920753    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85416faf_a9e8_4d36_8187_35b65fa07f4d.slice/crio-793e81577d370524264803fb372e4119346919fa592706a93e3db7197b7cecee WatchSource:0}: Error finding container 793e81577d370524264803fb372e4119346919fa592706a93e3db7197b7cecee: Status 404 returned error can't find the container with id 793e81577d370524264803fb372e4119346919fa592706a93e3db7197b7cecee
Jul 16 23:02:43.640822 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:43.640780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" event={"ID":"85416faf-a9e8-4d36-8187-35b65fa07f4d","Type":"ContainerStarted","Data":"3c428981c17cd45d52691cd51d6e24feb284ed9f1208193e49d91e969394ed6c"}
Jul 16 23:02:43.641232 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:43.640844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" event={"ID":"85416faf-a9e8-4d36-8187-35b65fa07f4d","Type":"ContainerStarted","Data":"793e81577d370524264803fb372e4119346919fa592706a93e3db7197b7cecee"}
Jul 16 23:02:43.657119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:43.657057    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" podStartSLOduration=1.657040574 podStartE2EDuration="1.657040574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:02:43.655118156 +0000 UTC m=+2719.975674229" watchObservedRunningTime="2026-07-16 23:02:43.657040574 +0000 UTC m=+2719.977596700"
Jul 16 23:02:44.648339 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:44.648304    2564 generic.go:356] "Generic (PLEG): container finished" podID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerID="3c428981c17cd45d52691cd51d6e24feb284ed9f1208193e49d91e969394ed6c" exitCode=6
Jul 16 23:02:44.648765 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:44.648390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" event={"ID":"85416faf-a9e8-4d36-8187-35b65fa07f4d","Type":"ContainerDied","Data":"3c428981c17cd45d52691cd51d6e24feb284ed9f1208193e49d91e969394ed6c"}
Jul 16 23:02:44.648765 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:44.648603    2564 scope.go:117] "RemoveContainer" containerID="3c428981c17cd45d52691cd51d6e24feb284ed9f1208193e49d91e969394ed6c"
Jul 16 23:02:45.656397 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:45.656317    2564 generic.go:356] "Generic (PLEG): container finished" podID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerID="0796ab7a343420f29ef94c3cae013b18f3730563ee52f2009dd4a67e73b43ece" exitCode=6
Jul 16 23:02:45.656397 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:45.656357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" event={"ID":"85416faf-a9e8-4d36-8187-35b65fa07f4d","Type":"ContainerDied","Data":"0796ab7a343420f29ef94c3cae013b18f3730563ee52f2009dd4a67e73b43ece"}
Jul 16 23:02:45.656877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:45.656408    2564 scope.go:117] "RemoveContainer" containerID="3c428981c17cd45d52691cd51d6e24feb284ed9f1208193e49d91e969394ed6c"
Jul 16 23:02:45.656877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:45.656670    2564 scope.go:117] "RemoveContainer" containerID="0796ab7a343420f29ef94c3cae013b18f3730563ee52f2009dd4a67e73b43ece"
Jul 16 23:02:45.656961 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:45.656910    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt_odh-ai-gateway-infra(85416faf-a9e8-4d36-8187-35b65fa07f4d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d"
Jul 16 23:02:46.598321 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:46.598279    2564 patch_prober.go:28] interesting pod/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 23:02:46.598532 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:46.598359    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 23:02:46.665829 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:46.665799    2564 scope.go:117] "RemoveContainer" containerID="0796ab7a343420f29ef94c3cae013b18f3730563ee52f2009dd4a67e73b43ece"
Jul 16 23:02:46.679366 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:46.666032    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt_odh-ai-gateway-infra(85416faf-a9e8-4d36-8187-35b65fa07f4d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d"
Jul 16 23:02:47.673971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:47.673925    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerID="e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6" exitCode=0
Jul 16 23:02:47.674481 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:47.674058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" event={"ID":"5f494fee-5fb7-45e4-94dd-e2343411ca37","Type":"ContainerDied","Data":"e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"}
Jul 16 23:02:47.674548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:47.674530    2564 scope.go:117] "RemoveContainer" containerID="e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"
Jul 16 23:02:47.676171 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:47.676108    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:47.676289 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:47.676224    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:47.676289 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:47.676271    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:48.685676 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:48.685635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" event={"ID":"5f494fee-5fb7-45e4-94dd-e2343411ca37","Type":"ContainerStarted","Data":"eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6"}
Jul 16 23:02:49.251781 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:49.251738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:49.252025 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:49.252005    2564 patch_prober.go:28] interesting pod/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Jul 16 23:02:49.252079 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:49.252050    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Jul 16 23:02:49.987948 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:49.987901    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:02:50.251845 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:50.251768    2564 patch_prober.go:28] interesting pod/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Jul 16 23:02:50.251975 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:50.251832    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Jul 16 23:02:51.251715 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:51.251669    2564 patch_prober.go:28] interesting pod/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Jul 16 23:02:51.252119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:51.251746    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Jul 16 23:02:52.251157 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:52.251086    2564 patch_prober.go:28] interesting pod/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Jul 16 23:02:52.251361 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:52.251187    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Jul 16 23:02:53.308955 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.308921    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:53.309468 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.309348    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:53.310154 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.310120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:02:53.719880 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.719844    2564 generic.go:356] "Generic (PLEG): container finished" podID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerID="39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558" exitCode=0
Jul 16 23:02:53.720100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.719914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" event={"ID":"a9cfefab-0cb2-488b-af0e-def2b67261b5","Type":"ContainerDied","Data":"39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"}
Jul 16 23:02:53.720368 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.720355    2564 scope.go:117] "RemoveContainer" containerID="39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"
Jul 16 23:02:53.721401 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.721375    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:53.721475 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.721458    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:53.721522 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:53.721509    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:02:54.618980 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:54.618938    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:54.726959 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:54.726921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" event={"ID":"a9cfefab-0cb2-488b-af0e-def2b67261b5","Type":"ContainerStarted","Data":"52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b"}
Jul 16 23:02:54.835621 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:54.835579    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:54.836697 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:54.836672    2564 patch_prober.go:28] interesting pod/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Jul 16 23:02:54.836822 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:54.836720    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Jul 16 23:02:55.835550 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:55.835508    2564 patch_prober.go:28] interesting pod/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Jul 16 23:02:55.835939 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:55.835582    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Jul 16 23:02:56.835853 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:56.835811    2564 patch_prober.go:28] interesting pod/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Jul 16 23:02:56.836352 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:56.835871    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Jul 16 23:02:57.596325 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:57.596287    2564 scope.go:117] "RemoveContainer" containerID="0796ab7a343420f29ef94c3cae013b18f3730563ee52f2009dd4a67e73b43ece"
Jul 16 23:02:57.835764 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:57.835728    2564 patch_prober.go:28] interesting pod/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Jul 16 23:02:57.835931 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:57.835782    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Jul 16 23:02:58.749813 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.749723    2564 generic.go:356] "Generic (PLEG): container finished" podID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerID="0518c419e11fa976ca63bcaab5feaab77941fff20c8a15c765258c51ad13df69" exitCode=6
Jul 16 23:02:58.749813 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.749795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" event={"ID":"85416faf-a9e8-4d36-8187-35b65fa07f4d","Type":"ContainerDied","Data":"0518c419e11fa976ca63bcaab5feaab77941fff20c8a15c765258c51ad13df69"}
Jul 16 23:02:58.750003 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.749886    2564 scope.go:117] "RemoveContainer" containerID="0796ab7a343420f29ef94c3cae013b18f3730563ee52f2009dd4a67e73b43ece"
Jul 16 23:02:58.750190 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.750175    2564 scope.go:117] "RemoveContainer" containerID="0518c419e11fa976ca63bcaab5feaab77941fff20c8a15c765258c51ad13df69"
Jul 16 23:02:58.750425 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:02:58.750404    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt_odh-ai-gateway-infra(85416faf-a9e8-4d36-8187-35b65fa07f4d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d"
Jul 16 23:02:58.839687 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.839656    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:58.840065 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.839928    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:58.840695 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:58.840674    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:02:59.778526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:59.778491    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"]
Jul 16 23:02:59.910178 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:02:59.910152    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:03:00.022659 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.022623    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85416faf-a9e8-4d36-8187-35b65fa07f4d-maas-api-service-ca\") pod \"85416faf-a9e8-4d36-8187-35b65fa07f4d\" (UID: \"85416faf-a9e8-4d36-8187-35b65fa07f4d\") "
Jul 16 23:03:00.022974 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.022952    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85416faf-a9e8-4d36-8187-35b65fa07f4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85416faf-a9e8-4d36-8187-35b65fa07f4d" (UID: "85416faf-a9e8-4d36-8187-35b65fa07f4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:03:00.123691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.123604    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85416faf-a9e8-4d36-8187-35b65fa07f4d-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:03:00.766890 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.766862    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"
Jul 16 23:03:00.767090 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.766892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt" event={"ID":"85416faf-a9e8-4d36-8187-35b65fa07f4d","Type":"ContainerDied","Data":"793e81577d370524264803fb372e4119346919fa592706a93e3db7197b7cecee"}
Jul 16 23:03:00.767090 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.766948    2564 scope.go:117] "RemoveContainer" containerID="0518c419e11fa976ca63bcaab5feaab77941fff20c8a15c765258c51ad13df69"
Jul 16 23:03:00.790013 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.789982    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"]
Jul 16 23:03:00.795877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:00.795850    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-tj2zt"]
Jul 16 23:03:01.601355 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:01.601325    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" path="/var/lib/kubelet/pods/85416faf-a9e8-4d36-8187-35b65fa07f4d/volumes"
Jul 16 23:03:17.118465 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118432    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"]
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118787    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118798    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f857a59-5b45-416f-a87a-8b2b70442b6e" containerName="revoke-keys"
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118812    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118817    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118825    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118830    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:17.118898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118899    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:17.119127 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.118907    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:17.121476 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.121461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:17.123905 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.123886    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:03:17.123905 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.123893    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:03:17.130158 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.130119    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"]
Jul 16 23:03:17.285616 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.285568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a5064-4a79-4446-913e-c78e4c95b897-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5\" (UID: \"c45a5064-4a79-4446-913e-c78e4c95b897\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:17.386969 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.386885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a5064-4a79-4446-913e-c78e4c95b897-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5\" (UID: \"c45a5064-4a79-4446-913e-c78e4c95b897\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:17.387506 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.387487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a5064-4a79-4446-913e-c78e4c95b897-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5\" (UID: \"c45a5064-4a79-4446-913e-c78e4c95b897\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:17.433847 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.433815    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:17.588701 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.588676    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"]
Jul 16 23:03:17.590775 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:03:17.590741    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45a5064_4a79_4446_913e_c78e4c95b897.slice/crio-2a6154c432d42120fc9d026b5f4d3420d05e1cb02149ec8b3486636f2cf7ea28 WatchSource:0}: Error finding container 2a6154c432d42120fc9d026b5f4d3420d05e1cb02149ec8b3486636f2cf7ea28: Status 404 returned error can't find the container with id 2a6154c432d42120fc9d026b5f4d3420d05e1cb02149ec8b3486636f2cf7ea28
Jul 16 23:03:17.868722 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:17.868690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" event={"ID":"c45a5064-4a79-4446-913e-c78e4c95b897","Type":"ContainerStarted","Data":"2a6154c432d42120fc9d026b5f4d3420d05e1cb02149ec8b3486636f2cf7ea28"}
Jul 16 23:03:18.396092 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:18.396056    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:03:18.875409 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:18.875379    2564 generic.go:356] "Generic (PLEG): container finished" podID="c45a5064-4a79-4446-913e-c78e4c95b897" containerID="c359c1b7cc30cc60d2de678df9ca244f26600353e7b551c13f10536311d05157" exitCode=6
Jul 16 23:03:18.875565 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:18.875463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" event={"ID":"c45a5064-4a79-4446-913e-c78e4c95b897","Type":"ContainerDied","Data":"c359c1b7cc30cc60d2de678df9ca244f26600353e7b551c13f10536311d05157"}
Jul 16 23:03:18.875628 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:18.875616    2564 scope.go:117] "RemoveContainer" containerID="c359c1b7cc30cc60d2de678df9ca244f26600353e7b551c13f10536311d05157"
Jul 16 23:03:19.883054 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:19.882967    2564 generic.go:356] "Generic (PLEG): container finished" podID="c45a5064-4a79-4446-913e-c78e4c95b897" containerID="0036be368d3f9edb49658f826dc0a71584d2e7ca215ed33e8ec2f6a6b5653251" exitCode=6
Jul 16 23:03:19.883054 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:19.883004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" event={"ID":"c45a5064-4a79-4446-913e-c78e4c95b897","Type":"ContainerDied","Data":"0036be368d3f9edb49658f826dc0a71584d2e7ca215ed33e8ec2f6a6b5653251"}
Jul 16 23:03:19.883527 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:19.883067    2564 scope.go:117] "RemoveContainer" containerID="c359c1b7cc30cc60d2de678df9ca244f26600353e7b551c13f10536311d05157"
Jul 16 23:03:19.883527 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:19.883295    2564 scope.go:117] "RemoveContainer" containerID="0036be368d3f9edb49658f826dc0a71584d2e7ca215ed33e8ec2f6a6b5653251"
Jul 16 23:03:19.883527 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:19.883490    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5_odh-ai-gateway-infra(c45a5064-4a79-4446-913e-c78e4c95b897)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" podUID="c45a5064-4a79-4446-913e-c78e4c95b897"
Jul 16 23:03:20.890359 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:20.890332    2564 scope.go:117] "RemoveContainer" containerID="0036be368d3f9edb49658f826dc0a71584d2e7ca215ed33e8ec2f6a6b5653251"
Jul 16 23:03:20.890755 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:20.890507    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5_odh-ai-gateway-infra(c45a5064-4a79-4446-913e-c78e4c95b897)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" podUID="c45a5064-4a79-4446-913e-c78e4c95b897"
Jul 16 23:03:31.883583 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:31.883549    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:03:32.458386 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.458351    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"]
Jul 16 23:03:32.458719 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.458706    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:32.458764 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.458720    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:32.458816 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.458785    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85416faf-a9e8-4d36-8187-35b65fa07f4d" containerName="revoke-keys"
Jul 16 23:03:32.461128 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.461113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:32.468218 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.468175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"]
Jul 16 23:03:32.517440 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.517396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf\" (UID: \"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:32.618264 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.618230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf\" (UID: \"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:32.618868 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.618846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf\" (UID: \"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:32.774419 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.774316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:32.907062 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.907033    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"]
Jul 16 23:03:32.908278 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:03:32.908248    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9eb7b7_7bce_482e_9f1d_fefdd6fd34a5.slice/crio-170be7525a8aebe9040d6d1f74625bc1d9cb27aabf6883081b557a3dbb9f0369 WatchSource:0}: Error finding container 170be7525a8aebe9040d6d1f74625bc1d9cb27aabf6883081b557a3dbb9f0369: Status 404 returned error can't find the container with id 170be7525a8aebe9040d6d1f74625bc1d9cb27aabf6883081b557a3dbb9f0369
Jul 16 23:03:32.959497 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:32.959460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" event={"ID":"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5","Type":"ContainerStarted","Data":"170be7525a8aebe9040d6d1f74625bc1d9cb27aabf6883081b557a3dbb9f0369"}
Jul 16 23:03:33.966498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:33.966462    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerID="cc98dc984cd3e387bf1e4368430d8bd4df4dd3a9d5cb67cad125e8fb40f852d7" exitCode=6
Jul 16 23:03:33.966913 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:33.966545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" event={"ID":"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5","Type":"ContainerDied","Data":"cc98dc984cd3e387bf1e4368430d8bd4df4dd3a9d5cb67cad125e8fb40f852d7"}
Jul 16 23:03:33.966913 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:33.966739    2564 scope.go:117] "RemoveContainer" containerID="cc98dc984cd3e387bf1e4368430d8bd4df4dd3a9d5cb67cad125e8fb40f852d7"
Jul 16 23:03:34.596762 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:34.596679    2564 scope.go:117] "RemoveContainer" containerID="0036be368d3f9edb49658f826dc0a71584d2e7ca215ed33e8ec2f6a6b5653251"
Jul 16 23:03:34.974591 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:34.974561    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerID="69af9f7c772a447879f82e37be3f742e2acf739d5e48e964db949d61c1fda257" exitCode=6
Jul 16 23:03:34.975056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:34.974628    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" event={"ID":"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5","Type":"ContainerDied","Data":"69af9f7c772a447879f82e37be3f742e2acf739d5e48e964db949d61c1fda257"}
Jul 16 23:03:34.975056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:34.974693    2564 scope.go:117] "RemoveContainer" containerID="cc98dc984cd3e387bf1e4368430d8bd4df4dd3a9d5cb67cad125e8fb40f852d7"
Jul 16 23:03:34.975056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:34.974937    2564 scope.go:117] "RemoveContainer" containerID="69af9f7c772a447879f82e37be3f742e2acf739d5e48e964db949d61c1fda257"
Jul 16 23:03:34.975242 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:34.975175    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf_odh-ai-gateway-infra(dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5"
Jul 16 23:03:34.977192 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:34.977170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" event={"ID":"c45a5064-4a79-4446-913e-c78e4c95b897","Type":"ContainerStarted","Data":"6adb7d5de4225014ac8255c2132629a2a9cdaaa17b822e8345d2e59040a7191c"}
Jul 16 23:03:35.007669 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:35.007606    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" podStartSLOduration=18.007591041 podStartE2EDuration="18.007591041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:03:35.006700013 +0000 UTC m=+2771.327256085" watchObservedRunningTime="2026-07-16 23:03:35.007591041 +0000 UTC m=+2771.328147112"
Jul 16 23:03:35.985056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:35.985028    2564 scope.go:117] "RemoveContainer" containerID="69af9f7c772a447879f82e37be3f742e2acf739d5e48e964db949d61c1fda257"
Jul 16 23:03:35.985590 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:35.985395    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf_odh-ai-gateway-infra(dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5"
Jul 16 23:03:35.986592 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:35.986565    2564 generic.go:356] "Generic (PLEG): container finished" podID="c45a5064-4a79-4446-913e-c78e4c95b897" containerID="6adb7d5de4225014ac8255c2132629a2a9cdaaa17b822e8345d2e59040a7191c" exitCode=6
Jul 16 23:03:35.986717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:35.986641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" event={"ID":"c45a5064-4a79-4446-913e-c78e4c95b897","Type":"ContainerDied","Data":"6adb7d5de4225014ac8255c2132629a2a9cdaaa17b822e8345d2e59040a7191c"}
Jul 16 23:03:35.986717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:35.986694    2564 scope.go:117] "RemoveContainer" containerID="0036be368d3f9edb49658f826dc0a71584d2e7ca215ed33e8ec2f6a6b5653251"
Jul 16 23:03:35.986927 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:35.986914    2564 scope.go:117] "RemoveContainer" containerID="6adb7d5de4225014ac8255c2132629a2a9cdaaa17b822e8345d2e59040a7191c"
Jul 16 23:03:35.987128 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:35.987106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5_odh-ai-gateway-infra(c45a5064-4a79-4446-913e-c78e4c95b897)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" podUID="c45a5064-4a79-4446-913e-c78e4c95b897"
Jul 16 23:03:36.017361 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:36.017327    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"]
Jul 16 23:03:37.125259 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:37.125238    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:37.158112 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:37.158082    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a5064-4a79-4446-913e-c78e4c95b897-maas-api-service-ca\") pod \"c45a5064-4a79-4446-913e-c78e4c95b897\" (UID: \"c45a5064-4a79-4446-913e-c78e4c95b897\") "
Jul 16 23:03:37.158438 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:37.158417    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c45a5064-4a79-4446-913e-c78e4c95b897-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c45a5064-4a79-4446-913e-c78e4c95b897" (UID: "c45a5064-4a79-4446-913e-c78e4c95b897"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:03:37.259805 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:37.259716    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a5064-4a79-4446-913e-c78e4c95b897-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:03:38.000874 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:38.000843    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5" event={"ID":"c45a5064-4a79-4446-913e-c78e4c95b897","Type":"ContainerDied","Data":"2a6154c432d42120fc9d026b5f4d3420d05e1cb02149ec8b3486636f2cf7ea28"}
Jul 16 23:03:38.000874 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:38.000878    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"
Jul 16 23:03:38.001109 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:38.000904    2564 scope.go:117] "RemoveContainer" containerID="6adb7d5de4225014ac8255c2132629a2a9cdaaa17b822e8345d2e59040a7191c"
Jul 16 23:03:38.018287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:38.018259    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"]
Jul 16 23:03:38.021651 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:38.021629    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-5l6p5"]
Jul 16 23:03:39.601948 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:39.601918    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" path="/var/lib/kubelet/pods/c45a5064-4a79-4446-913e-c78e4c95b897/volumes"
Jul 16 23:03:47.482456 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:47.482364    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:03:49.039101 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039063    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"]
Jul 16 23:03:49.039498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039422    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039433    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039448    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039453    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039459    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039465    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039513    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.039691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.039521    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:03:49.042458 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.042441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:03:49.050531 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.050500    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"]
Jul 16 23:03:49.162601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.162569    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb529f-1c37-4e48-a624-dcb5cc317fd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rndqs\" (UID: \"0bcb529f-1c37-4e48-a624-dcb5cc317fd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:03:49.263071 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.263031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb529f-1c37-4e48-a624-dcb5cc317fd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rndqs\" (UID: \"0bcb529f-1c37-4e48-a624-dcb5cc317fd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:03:49.263667 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.263648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb529f-1c37-4e48-a624-dcb5cc317fd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rndqs\" (UID: \"0bcb529f-1c37-4e48-a624-dcb5cc317fd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:03:49.354436 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.354354    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:03:49.486652 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:49.486628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"]
Jul 16 23:03:49.489258 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:03:49.489229    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcb529f_1c37_4e48_a624_dcb5cc317fd8.slice/crio-9b74b35d6f7399ca5f009df4f010d09053434329a0f81a01f328463477f4eb1d WatchSource:0}: Error finding container 9b74b35d6f7399ca5f009df4f010d09053434329a0f81a01f328463477f4eb1d: Status 404 returned error can't find the container with id 9b74b35d6f7399ca5f009df4f010d09053434329a0f81a01f328463477f4eb1d
Jul 16 23:03:50.066949 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:50.066914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerStarted","Data":"3856aada5be3e2c00b07c9a99d87ebdb333678f23f68d88f0b0528c49e8363e4"}
Jul 16 23:03:50.067446 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:50.066970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerStarted","Data":"9b74b35d6f7399ca5f009df4f010d09053434329a0f81a01f328463477f4eb1d"}
Jul 16 23:03:50.084371 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:50.084318    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" podStartSLOduration=1.084297784 podStartE2EDuration="1.084297784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:03:50.082865621 +0000 UTC m=+2786.403421693" watchObservedRunningTime="2026-07-16 23:03:50.084297784 +0000 UTC m=+2786.404853856"
Jul 16 23:03:50.596278 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:50.596243    2564 scope.go:117] "RemoveContainer" containerID="69af9f7c772a447879f82e37be3f742e2acf739d5e48e964db949d61c1fda257"
Jul 16 23:03:51.075551 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:51.075521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" event={"ID":"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5","Type":"ContainerStarted","Data":"fbf7b9356f1cb06a0dbfb340745a07b52c51f7338ec476f710b4b386e0368660"}
Jul 16 23:03:51.076982 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:51.076957    2564 generic.go:356] "Generic (PLEG): container finished" podID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerID="3856aada5be3e2c00b07c9a99d87ebdb333678f23f68d88f0b0528c49e8363e4" exitCode=6
Jul 16 23:03:51.077102 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:51.077016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerDied","Data":"3856aada5be3e2c00b07c9a99d87ebdb333678f23f68d88f0b0528c49e8363e4"}
Jul 16 23:03:51.077182 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:51.077162    2564 scope.go:117] "RemoveContainer" containerID="3856aada5be3e2c00b07c9a99d87ebdb333678f23f68d88f0b0528c49e8363e4"
Jul 16 23:03:51.096117 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:51.096074    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" podStartSLOduration=19.096058331 podStartE2EDuration="19.096058331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:03:51.094327616 +0000 UTC m=+2787.414883689" watchObservedRunningTime="2026-07-16 23:03:51.096058331 +0000 UTC m=+2787.416614403"
Jul 16 23:03:52.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.084294    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerID="fbf7b9356f1cb06a0dbfb340745a07b52c51f7338ec476f710b4b386e0368660" exitCode=6
Jul 16 23:03:52.084729 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.084376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" event={"ID":"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5","Type":"ContainerDied","Data":"fbf7b9356f1cb06a0dbfb340745a07b52c51f7338ec476f710b4b386e0368660"}
Jul 16 23:03:52.084729 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.084449    2564 scope.go:117] "RemoveContainer" containerID="69af9f7c772a447879f82e37be3f742e2acf739d5e48e964db949d61c1fda257"
Jul 16 23:03:52.084871 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.084772    2564 scope.go:117] "RemoveContainer" containerID="fbf7b9356f1cb06a0dbfb340745a07b52c51f7338ec476f710b4b386e0368660"
Jul 16 23:03:52.085053 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:52.085029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf_odh-ai-gateway-infra(dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5"
Jul 16 23:03:52.086361 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.086340    2564 generic.go:356] "Generic (PLEG): container finished" podID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerID="de5943a1f453d5b70df3119408ed83aeddb2391c5846092bdccb8b05d1be263f" exitCode=6
Jul 16 23:03:52.086452 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.086399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerDied","Data":"de5943a1f453d5b70df3119408ed83aeddb2391c5846092bdccb8b05d1be263f"}
Jul 16 23:03:52.086586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.086573    2564 scope.go:117] "RemoveContainer" containerID="de5943a1f453d5b70df3119408ed83aeddb2391c5846092bdccb8b05d1be263f"
Jul 16 23:03:52.086789 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:52.086760    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rndqs_odh-ai-gateway-infra(0bcb529f-1c37-4e48-a624-dcb5cc317fd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8"
Jul 16 23:03:52.094835 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.094821    2564 scope.go:117] "RemoveContainer" containerID="3856aada5be3e2c00b07c9a99d87ebdb333678f23f68d88f0b0528c49e8363e4"
Jul 16 23:03:52.115691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:52.115646    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"]
Jul 16 23:03:53.096333 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:53.096306    2564 scope.go:117] "RemoveContainer" containerID="de5943a1f453d5b70df3119408ed83aeddb2391c5846092bdccb8b05d1be263f"
Jul 16 23:03:53.096827 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:03:53.096504    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rndqs_odh-ai-gateway-infra(0bcb529f-1c37-4e48-a624-dcb5cc317fd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8"
Jul 16 23:03:53.224804 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:53.224779    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:53.303781 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:53.303742    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5-maas-api-service-ca\") pod \"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5\" (UID: \"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5\") "
Jul 16 23:03:53.304176 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:53.304127    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" (UID: "dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:03:53.405450 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:53.405359    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:03:54.103117 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:54.103085    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"
Jul 16 23:03:54.103586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:54.103127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf" event={"ID":"dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5","Type":"ContainerDied","Data":"170be7525a8aebe9040d6d1f74625bc1d9cb27aabf6883081b557a3dbb9f0369"}
Jul 16 23:03:54.103586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:54.103222    2564 scope.go:117] "RemoveContainer" containerID="fbf7b9356f1cb06a0dbfb340745a07b52c51f7338ec476f710b4b386e0368660"
Jul 16 23:03:54.119632 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:54.119606    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"]
Jul 16 23:03:54.127782 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:54.127759    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-54zrf"]
Jul 16 23:03:55.602338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:03:55.602305    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" path="/var/lib/kubelet/pods/dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5/volumes"
Jul 16 23:04:02.276053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:02.276018    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:04:04.596818 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.596790    2564 scope.go:117] "RemoveContainer" containerID="de5943a1f453d5b70df3119408ed83aeddb2391c5846092bdccb8b05d1be263f"
Jul 16 23:04:04.775979 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.775952    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"]
Jul 16 23:04:04.776318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776304    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:04.776367 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776319    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:04.776367 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776333    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:04.776367 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776339    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:04.776464 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776397    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:04.776464 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776406    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c45a5064-4a79-4446-913e-c78e4c95b897" containerName="revoke-keys"
Jul 16 23:04:04.776464 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.776413    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:04.778958 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.778940    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:04.787767 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.787741    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"]
Jul 16 23:04:04.901351 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:04.901298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63425147-0514-4b6d-9ecf-e28d90b6a908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp\" (UID: \"63425147-0514-4b6d-9ecf-e28d90b6a908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:05.002506 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:05.002471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63425147-0514-4b6d-9ecf-e28d90b6a908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp\" (UID: \"63425147-0514-4b6d-9ecf-e28d90b6a908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:05.003116 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:05.003095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63425147-0514-4b6d-9ecf-e28d90b6a908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp\" (UID: \"63425147-0514-4b6d-9ecf-e28d90b6a908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:05.094602 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:05.094555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:05.165919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:05.165874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerStarted","Data":"74ddb29e93f524d515f9c0b6414364b21acc895af6435111e2bc4113dab23284"}
Jul 16 23:04:05.231846 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:05.231822    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"]
Jul 16 23:04:05.241186 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:04:05.241154    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63425147_0514_4b6d_9ecf_e28d90b6a908.slice/crio-74cb1ae585efdb51c918cdd6f0f39d455915f4474205e7b18704d71e567e73e3 WatchSource:0}: Error finding container 74cb1ae585efdb51c918cdd6f0f39d455915f4474205e7b18704d71e567e73e3: Status 404 returned error can't find the container with id 74cb1ae585efdb51c918cdd6f0f39d455915f4474205e7b18704d71e567e73e3
Jul 16 23:04:06.173525 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.173491    2564 generic.go:356] "Generic (PLEG): container finished" podID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerID="74ddb29e93f524d515f9c0b6414364b21acc895af6435111e2bc4113dab23284" exitCode=6
Jul 16 23:04:06.173959 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.173564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerDied","Data":"74ddb29e93f524d515f9c0b6414364b21acc895af6435111e2bc4113dab23284"}
Jul 16 23:04:06.173959 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.173633    2564 scope.go:117] "RemoveContainer" containerID="de5943a1f453d5b70df3119408ed83aeddb2391c5846092bdccb8b05d1be263f"
Jul 16 23:04:06.173959 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.173925    2564 scope.go:117] "RemoveContainer" containerID="74ddb29e93f524d515f9c0b6414364b21acc895af6435111e2bc4113dab23284"
Jul 16 23:04:06.174249 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:04:06.174211    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rndqs_odh-ai-gateway-infra(0bcb529f-1c37-4e48-a624-dcb5cc317fd8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8"
Jul 16 23:04:06.175437 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.175414    2564 generic.go:356] "Generic (PLEG): container finished" podID="63425147-0514-4b6d-9ecf-e28d90b6a908" containerID="a767e24b2ac53297d00ef246913c6f382ac8c3797db32bbc2c9f8eb23bc7a039" exitCode=0
Jul 16 23:04:06.175542 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.175485    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp" event={"ID":"63425147-0514-4b6d-9ecf-e28d90b6a908","Type":"ContainerDied","Data":"a767e24b2ac53297d00ef246913c6f382ac8c3797db32bbc2c9f8eb23bc7a039"}
Jul 16 23:04:06.175542 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.175535    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp" event={"ID":"63425147-0514-4b6d-9ecf-e28d90b6a908","Type":"ContainerStarted","Data":"74cb1ae585efdb51c918cdd6f0f39d455915f4474205e7b18704d71e567e73e3"}
Jul 16 23:04:06.190705 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:06.190683    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"]
Jul 16 23:04:07.318794 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.318760    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:04:07.344585 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.344563    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:07.423665 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.423632    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63425147-0514-4b6d-9ecf-e28d90b6a908-maas-api-service-ca\") pod \"63425147-0514-4b6d-9ecf-e28d90b6a908\" (UID: \"63425147-0514-4b6d-9ecf-e28d90b6a908\") "
Jul 16 23:04:07.423839 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.423685    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb529f-1c37-4e48-a624-dcb5cc317fd8-maas-api-service-ca\") pod \"0bcb529f-1c37-4e48-a624-dcb5cc317fd8\" (UID: \"0bcb529f-1c37-4e48-a624-dcb5cc317fd8\") "
Jul 16 23:04:07.423985 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.423964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63425147-0514-4b6d-9ecf-e28d90b6a908-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63425147-0514-4b6d-9ecf-e28d90b6a908" (UID: "63425147-0514-4b6d-9ecf-e28d90b6a908"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:04:07.424050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.424032    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb529f-1c37-4e48-a624-dcb5cc317fd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0bcb529f-1c37-4e48-a624-dcb5cc317fd8" (UID: "0bcb529f-1c37-4e48-a624-dcb5cc317fd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:04:07.525151 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.525090    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63425147-0514-4b6d-9ecf-e28d90b6a908-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:07.525151 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:07.525126    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb529f-1c37-4e48-a624-dcb5cc317fd8-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:08.189627 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.189599    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"
Jul 16 23:04:08.189798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.189647    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs" event={"ID":"0bcb529f-1c37-4e48-a624-dcb5cc317fd8","Type":"ContainerDied","Data":"9b74b35d6f7399ca5f009df4f010d09053434329a0f81a01f328463477f4eb1d"}
Jul 16 23:04:08.189798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.189712    2564 scope.go:117] "RemoveContainer" containerID="74ddb29e93f524d515f9c0b6414364b21acc895af6435111e2bc4113dab23284"
Jul 16 23:04:08.191065 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.190941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp" event={"ID":"63425147-0514-4b6d-9ecf-e28d90b6a908","Type":"ContainerDied","Data":"74cb1ae585efdb51c918cdd6f0f39d455915f4474205e7b18704d71e567e73e3"}
Jul 16 23:04:08.191065 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.190983    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74cb1ae585efdb51c918cdd6f0f39d455915f4474205e7b18704d71e567e73e3"
Jul 16 23:04:08.191065 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.191002    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp"
Jul 16 23:04:08.212231 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.208541    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"]
Jul 16 23:04:08.215938 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:08.215910    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rndqs"]
Jul 16 23:04:09.601960 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:09.601931    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" path="/var/lib/kubelet/pods/0bcb529f-1c37-4e48-a624-dcb5cc317fd8/volumes"
Jul 16 23:04:17.989543 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:17.989500    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:04:30.564123 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:30.564087    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"]
Jul 16 23:04:30.564668 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:30.564318    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy" containerID="cri-o://eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6" gracePeriod=30
Jul 16 23:04:35.824918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.824859    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:04:35.895749 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895713    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-data\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.895927 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895775    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5txp\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-kube-api-access-r5txp\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.895927 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895803    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-certs\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.895927 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895845    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-podinfo\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.895927 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895899    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f494fee-5fb7-45e4-94dd-e2343411ca37-istiod-ca-cert\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.896119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895933    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-envoy\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.896119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895961    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-socket\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.896119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.895999    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-credential-socket\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.896119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896044    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-token\") pod \"5f494fee-5fb7-45e4-94dd-e2343411ca37\" (UID: \"5f494fee-5fb7-45e4-94dd-e2343411ca37\") "
Jul 16 23:04:35.896373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896117    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:35.896373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896343    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:35.896491 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896401    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.896491 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896409    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f494fee-5fb7-45e4-94dd-e2343411ca37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:04:35.896673 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896648    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:35.896673 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.896664    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:35.898172 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.898146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-kube-api-access-r5txp" (OuterVolumeSpecName: "kube-api-access-r5txp") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "kube-api-access-r5txp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:04:35.898366 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.898347    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:04:35.898591 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.898566    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:04:35.898591 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.898575    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f494fee-5fb7-45e4-94dd-e2343411ca37" (UID: "5f494fee-5fb7-45e4-94dd-e2343411ca37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:35.997089 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997055    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f494fee-5fb7-45e4-94dd-e2343411ca37-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997089 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997085    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997089 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997094    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997089 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997102    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997406 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997111    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997406 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997119    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997406 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997127    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5txp\" (UniqueName: \"kubernetes.io/projected/5f494fee-5fb7-45e4-94dd-e2343411ca37-kube-api-access-r5txp\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:35.997406 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:35.997162    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f494fee-5fb7-45e4-94dd-e2343411ca37-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:36.348052 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.348015    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerID="eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6" exitCode=0
Jul 16 23:04:36.348277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.348066    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"
Jul 16 23:04:36.348277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.348115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" event={"ID":"5f494fee-5fb7-45e4-94dd-e2343411ca37","Type":"ContainerDied","Data":"eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6"}
Jul 16 23:04:36.348277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.348198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v" event={"ID":"5f494fee-5fb7-45e4-94dd-e2343411ca37","Type":"ContainerDied","Data":"03fcfec1716cf72ae4531af78b76d83216565940b6b687b0c2f324d9ae39315a"}
Jul 16 23:04:36.348277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.348227    2564 scope.go:117] "RemoveContainer" containerID="eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6"
Jul 16 23:04:36.358116 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.358097    2564 scope.go:117] "RemoveContainer" containerID="e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"
Jul 16 23:04:36.366787 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.366748    2564 scope.go:117] "RemoveContainer" containerID="eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6"
Jul 16 23:04:36.367343 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:04:36.367215    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6\": container with ID starting with eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6 not found: ID does not exist" containerID="eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6"
Jul 16 23:04:36.367343 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.367254    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6"} err="failed to get container status \"eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6\": rpc error: code = NotFound desc = could not find container \"eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6\": container with ID starting with eb74f1bede720df504c579ae9d4b9ce309c727fe38b017f8df814af5514c9dc6 not found: ID does not exist"
Jul 16 23:04:36.367343 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.367284    2564 scope.go:117] "RemoveContainer" containerID="e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"
Jul 16 23:04:36.367637 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:04:36.367619    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6\": container with ID starting with e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6 not found: ID does not exist" containerID="e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"
Jul 16 23:04:36.367677 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.367645    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6"} err="failed to get container status \"e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6\": rpc error: code = NotFound desc = could not find container \"e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6\": container with ID starting with e4332f4c422503a32a4958c33bc2239f35dfee8a408f410cec36929ce24d5eb6 not found: ID does not exist"
Jul 16 23:04:36.369989 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.369971    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"]
Jul 16 23:04:36.373977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:36.373954    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bf84b0-openshift-default-959cfdd5-2bw5v"]
Jul 16 23:04:37.601731 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:37.601699    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" path="/var/lib/kubelet/pods/5f494fee-5fb7-45e4-94dd-e2343411ca37/volumes"
Jul 16 23:04:39.083923 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.083889    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"]
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084249    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084261    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084276    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63425147-0514-4b6d-9ecf-e28d90b6a908" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084281    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63425147-0514-4b6d-9ecf-e28d90b6a908" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084291    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084296    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084302    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084308    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084322    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:39.084328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084327    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:39.084633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084378    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:39.084633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084385    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy"
Jul 16 23:04:39.084633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084390    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63425147-0514-4b6d-9ecf-e28d90b6a908" containerName="revoke-keys"
Jul 16 23:04:39.084633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084397    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9eb7b7-7bce-482e-9f1d-fefdd6fd34a5" containerName="revoke-keys"
Jul 16 23:04:39.084633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.084403    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:39.087677 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.087659    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:39.090032 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.090006    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:04:39.090173 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.090072    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:04:39.095554 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.095530    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"]
Jul 16 23:04:39.228541 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.228507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337f024-6fda-4e9b-8061-c1cfcb428306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rb65q\" (UID: \"2337f024-6fda-4e9b-8061-c1cfcb428306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:39.329555 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.329514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337f024-6fda-4e9b-8061-c1cfcb428306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rb65q\" (UID: \"2337f024-6fda-4e9b-8061-c1cfcb428306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:39.330114 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.330086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337f024-6fda-4e9b-8061-c1cfcb428306-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-rb65q\" (UID: \"2337f024-6fda-4e9b-8061-c1cfcb428306\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:39.399715 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.399623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:39.534984 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:39.534958    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"]
Jul 16 23:04:39.536827 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:04:39.536787    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2337f024_6fda_4e9b_8061_c1cfcb428306.slice/crio-6accab40b89f142858fe3482634525654d5725d6a4bdb4a20337c9d5578b0122 WatchSource:0}: Error finding container 6accab40b89f142858fe3482634525654d5725d6a4bdb4a20337c9d5578b0122: Status 404 returned error can't find the container with id 6accab40b89f142858fe3482634525654d5725d6a4bdb4a20337c9d5578b0122
Jul 16 23:04:40.373887 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:40.373846    2564 generic.go:356] "Generic (PLEG): container finished" podID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerID="ea3347896bb8107d1f3e97a6d4b3a4cfb26ac1f5bb75401d9383d30ae3db11e6" exitCode=6
Jul 16 23:04:40.374294 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:40.373912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" event={"ID":"2337f024-6fda-4e9b-8061-c1cfcb428306","Type":"ContainerDied","Data":"ea3347896bb8107d1f3e97a6d4b3a4cfb26ac1f5bb75401d9383d30ae3db11e6"}
Jul 16 23:04:40.374294 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:40.373964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" event={"ID":"2337f024-6fda-4e9b-8061-c1cfcb428306","Type":"ContainerStarted","Data":"6accab40b89f142858fe3482634525654d5725d6a4bdb4a20337c9d5578b0122"}
Jul 16 23:04:40.374294 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:40.374192    2564 scope.go:117] "RemoveContainer" containerID="ea3347896bb8107d1f3e97a6d4b3a4cfb26ac1f5bb75401d9383d30ae3db11e6"
Jul 16 23:04:41.381881 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:41.381789    2564 generic.go:356] "Generic (PLEG): container finished" podID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerID="15e212566c9a08d912e9f8ba381a46c463ec81c7bcd9b64aea02f54b83a18b16" exitCode=6
Jul 16 23:04:41.381881 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:41.381825    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" event={"ID":"2337f024-6fda-4e9b-8061-c1cfcb428306","Type":"ContainerDied","Data":"15e212566c9a08d912e9f8ba381a46c463ec81c7bcd9b64aea02f54b83a18b16"}
Jul 16 23:04:41.381881 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:41.381868    2564 scope.go:117] "RemoveContainer" containerID="ea3347896bb8107d1f3e97a6d4b3a4cfb26ac1f5bb75401d9383d30ae3db11e6"
Jul 16 23:04:41.382399 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:41.382098    2564 scope.go:117] "RemoveContainer" containerID="15e212566c9a08d912e9f8ba381a46c463ec81c7bcd9b64aea02f54b83a18b16"
Jul 16 23:04:41.382399 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:04:41.382348    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rb65q_odh-ai-gateway-infra(2337f024-6fda-4e9b-8061-c1cfcb428306)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306"
Jul 16 23:04:42.388977 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:42.388949    2564 scope.go:117] "RemoveContainer" containerID="15e212566c9a08d912e9f8ba381a46c463ec81c7bcd9b64aea02f54b83a18b16"
Jul 16 23:04:42.389379 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:04:42.389165    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rb65q_odh-ai-gateway-infra(2337f024-6fda-4e9b-8061-c1cfcb428306)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306"
Jul 16 23:04:44.480196 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:44.480161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:04:54.596769 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:54.596734    2564 scope.go:117] "RemoveContainer" containerID="15e212566c9a08d912e9f8ba381a46c463ec81c7bcd9b64aea02f54b83a18b16"
Jul 16 23:04:55.460417 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:55.460381    2564 generic.go:356] "Generic (PLEG): container finished" podID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerID="697be3340c11b5844239d6edd7cd73600f5b68f471f186c81318eca564257828" exitCode=6
Jul 16 23:04:55.460610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:55.460454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" event={"ID":"2337f024-6fda-4e9b-8061-c1cfcb428306","Type":"ContainerDied","Data":"697be3340c11b5844239d6edd7cd73600f5b68f471f186c81318eca564257828"}
Jul 16 23:04:55.460610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:55.460531    2564 scope.go:117] "RemoveContainer" containerID="15e212566c9a08d912e9f8ba381a46c463ec81c7bcd9b64aea02f54b83a18b16"
Jul 16 23:04:55.460856 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:55.460836    2564 scope.go:117] "RemoveContainer" containerID="697be3340c11b5844239d6edd7cd73600f5b68f471f186c81318eca564257828"
Jul 16 23:04:55.461256 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:04:55.461226    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-rb65q_odh-ai-gateway-infra(2337f024-6fda-4e9b-8061-c1cfcb428306)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306"
Jul 16 23:04:56.490437 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.490401    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"]
Jul 16 23:04:56.623986 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.623962    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:56.654534 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.654501    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"]
Jul 16 23:04:56.654778 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.654727    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy" containerID="cri-o://52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b" gracePeriod=30
Jul 16 23:04:56.784329 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.784236    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337f024-6fda-4e9b-8061-c1cfcb428306-maas-api-service-ca\") pod \"2337f024-6fda-4e9b-8061-c1cfcb428306\" (UID: \"2337f024-6fda-4e9b-8061-c1cfcb428306\") "
Jul 16 23:04:56.784653 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.784628    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2337f024-6fda-4e9b-8061-c1cfcb428306-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2337f024-6fda-4e9b-8061-c1cfcb428306" (UID: "2337f024-6fda-4e9b-8061-c1cfcb428306"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:04:56.885124 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:56.885085    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337f024-6fda-4e9b-8061-c1cfcb428306-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:04:57.473892 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.473855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q" event={"ID":"2337f024-6fda-4e9b-8061-c1cfcb428306","Type":"ContainerDied","Data":"6accab40b89f142858fe3482634525654d5725d6a4bdb4a20337c9d5578b0122"}
Jul 16 23:04:57.474074 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.473939    2564 scope.go:117] "RemoveContainer" containerID="697be3340c11b5844239d6edd7cd73600f5b68f471f186c81318eca564257828"
Jul 16 23:04:57.474074 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.473867    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"
Jul 16 23:04:57.501940 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.501894    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"]
Jul 16 23:04:57.504669 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.504647    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"]
Jul 16 23:04:57.505120 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505096    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505150    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505162    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505171    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505219    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505228    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505246    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy"
Jul 16 23:04:57.505266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505254    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy"
Jul 16 23:04:57.505599 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505355    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:04:57.505599 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505378    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb529f-1c37-4e48-a624-dcb5cc317fd8" containerName="revoke-keys"
Jul 16 23:04:57.505599 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505386    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:04:57.505599 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.505394    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f494fee-5fb7-45e4-94dd-e2343411ca37" containerName="istio-proxy"
Jul 16 23:04:57.509923 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.509905    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-rb65q"]
Jul 16 23:04:57.510034 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.509994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.513025 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.513007    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b9c909-openshift-default-dockercfg-m5sb6\""
Jul 16 23:04:57.525714 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.525677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"]
Jul 16 23:04:57.605014 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.604980    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" path="/var/lib/kubelet/pods/2337f024-6fda-4e9b-8061-c1cfcb428306/volumes"
Jul 16 23:04:57.693658 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693625    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-credential-socket\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.693844 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spkrb\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-kube-api-access-spkrb\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.693844 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-envoy\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.693844 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-podinfo\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.693844 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-socket\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.694008 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693857    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-data\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.694008 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-token\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.694008 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istiod-ca-cert\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.694008 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.693928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-certs\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.794865 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.794770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spkrb\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-kube-api-access-spkrb\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.794865 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.794816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-envoy\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.794931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-podinfo\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795001    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-socket\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795021    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-data\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-token\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istiod-ca-cert\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795379 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-certs\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795379 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-credential-socket\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795499 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795409    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-data\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795557 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795498    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-socket\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795596 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-credential-socket\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795744 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istiod-ca-cert\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.795895 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.795773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-certs\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.797313 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.797290    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-envoy\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.797472 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.797455    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-podinfo\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.803143 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.803111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-token\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.803325 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.803308    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spkrb\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-kube-api-access-spkrb\") pod \"e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") " pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.820338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.820312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:57.954952 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.954924    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"]
Jul 16 23:04:57.956886 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:04:57.956859    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8b4cbf_103b_4d45_97bc_af724a7e9f4f.slice/crio-28dc39864d02268b1705fbcb3aca8aec2cab69d45d6be773c9686a01e93c7f70 WatchSource:0}: Error finding container 28dc39864d02268b1705fbcb3aca8aec2cab69d45d6be773c9686a01e93c7f70: Status 404 returned error can't find the container with id 28dc39864d02268b1705fbcb3aca8aec2cab69d45d6be773c9686a01e93c7f70
Jul 16 23:04:57.959071 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.959040    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:04:57.959184 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.959123    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:04:57.959184 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:57.959179    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:04:58.481440 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:58.481381    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" event={"ID":"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f","Type":"ContainerStarted","Data":"7b363dccda85ab754d55386cf7be3d3d0aa49bf366e7b826fcc118c1cd90c7dd"}
Jul 16 23:04:58.481650 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:58.481464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" event={"ID":"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f","Type":"ContainerStarted","Data":"28dc39864d02268b1705fbcb3aca8aec2cab69d45d6be773c9686a01e93c7f70"}
Jul 16 23:04:58.503687 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:58.503627    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podStartSLOduration=1.5036107539999999 podStartE2EDuration="1.503610754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:04:58.50127962 +0000 UTC m=+2854.821835693" watchObservedRunningTime="2026-07-16 23:04:58.503610754 +0000 UTC m=+2854.824166826"
Jul 16 23:04:58.820457 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:58.820380    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:04:58.842381 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:58.842348    2564 patch_prober.go:28] interesting pod/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 23:04:58.842549 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:58.842402    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 23:04:59.826184 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:04:59.826148    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:00.494226 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:00.494198    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:00.495272 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:00.495251    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:01.910407 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:01.910378    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:05:02.034078 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cfefab-0cb2-488b-af0e-def2b67261b5-istiod-ca-cert\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034092    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-token\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034157    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-certs\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-envoy\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034198    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-credential-socket\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq6ck\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-kube-api-access-sq6ck\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034236    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-data\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034274    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-socket\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-podinfo\") pod \"a9cfefab-0cb2-488b-af0e-def2b67261b5\" (UID: \"a9cfefab-0cb2-488b-af0e-def2b67261b5\") "
Jul 16 23:05:02.034786 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034506    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:02.034786 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034618    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:02.034786 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034635    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9cfefab-0cb2-488b-af0e-def2b67261b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:05:02.034786 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034686    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.035004 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.034818    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:02.035400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.035368    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:02.036496 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.036471    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:05:02.036582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.036500    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:02.036582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.036521    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:05:02.036659 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.036608    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-kube-api-access-sq6ck" (OuterVolumeSpecName: "kube-api-access-sq6ck") pod "a9cfefab-0cb2-488b-af0e-def2b67261b5" (UID: "a9cfefab-0cb2-488b-af0e-def2b67261b5"). InnerVolumeSpecName "kube-api-access-sq6ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:05:02.135814 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135771    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cfefab-0cb2-488b-af0e-def2b67261b5-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.135814 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135801    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.135814 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135813    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.135814 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135823    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.136094 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135836    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq6ck\" (UniqueName: \"kubernetes.io/projected/a9cfefab-0cb2-488b-af0e-def2b67261b5-kube-api-access-sq6ck\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.136094 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135848    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.136094 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135861    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cfefab-0cb2-488b-af0e-def2b67261b5-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.136094 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.135872    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cfefab-0cb2-488b-af0e-def2b67261b5-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:02.506610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.506571    2564 generic.go:356] "Generic (PLEG): container finished" podID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerID="52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b" exitCode=0
Jul 16 23:05:02.506825 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.506619    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"
Jul 16 23:05:02.506825 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.506660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" event={"ID":"a9cfefab-0cb2-488b-af0e-def2b67261b5","Type":"ContainerDied","Data":"52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b"}
Jul 16 23:05:02.506825 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.506720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd" event={"ID":"a9cfefab-0cb2-488b-af0e-def2b67261b5","Type":"ContainerDied","Data":"88d82edfd1e2fe3114660e9fb3fbf08665ed330dc0baeb8537614b67e6685b66"}
Jul 16 23:05:02.506825 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.506749    2564 scope.go:117] "RemoveContainer" containerID="52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b"
Jul 16 23:05:02.516825 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.516809    2564 scope.go:117] "RemoveContainer" containerID="39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"
Jul 16 23:05:02.525387 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.525368    2564 scope.go:117] "RemoveContainer" containerID="52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b"
Jul 16 23:05:02.525650 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:05:02.525630    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b\": container with ID starting with 52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b not found: ID does not exist" containerID="52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b"
Jul 16 23:05:02.525726 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.525663    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b"} err="failed to get container status \"52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b\": rpc error: code = NotFound desc = could not find container \"52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b\": container with ID starting with 52a913f43305261e2ab29095aa865d2317d4ff68dc12264746781c19729ab29b not found: ID does not exist"
Jul 16 23:05:02.525726 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.525690    2564 scope.go:117] "RemoveContainer" containerID="39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"
Jul 16 23:05:02.525960 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:05:02.525942    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558\": container with ID starting with 39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558 not found: ID does not exist" containerID="39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"
Jul 16 23:05:02.526020 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.525970    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558"} err="failed to get container status \"39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558\": rpc error: code = NotFound desc = could not find container \"39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558\": container with ID starting with 39fecccb5c23774e634877eaf60937f5ae9f832f68f3b93272cd6186f1d56558 not found: ID does not exist"
Jul 16 23:05:02.542613 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.542585    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"]
Jul 16 23:05:02.549843 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:02.549812    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-30cb29-openshift-default-dfc59c7d5-kbqpd"]
Jul 16 23:05:03.601586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:03.601544    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" path="/var/lib/kubelet/pods/a9cfefab-0cb2-488b-af0e-def2b67261b5/volumes"
Jul 16 23:05:05.615417 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615385    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"]
Jul 16 23:05:05.615795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615735    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:05:05.615795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615745    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:05:05.615795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615755    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy"
Jul 16 23:05:05.615795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615760    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy"
Jul 16 23:05:05.615795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615767    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy"
Jul 16 23:05:05.615795 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615773    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy"
Jul 16 23:05:05.615990 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615823    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy"
Jul 16 23:05:05.615990 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.615832    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2337f024-6fda-4e9b-8061-c1cfcb428306" containerName="revoke-keys"
Jul 16 23:05:05.620316 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.620290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.622673 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.622649    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8c14db-openshift-default-dockercfg-fxlkb\""
Jul 16 23:05:05.631958 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.631929    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"]
Jul 16 23:05:05.665211 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pblk9\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-kube-api-access-pblk9\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665387 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665270    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-token\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665387 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-credential-socket\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665387 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istiod-ca-cert\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665506 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-envoy\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665506 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-certs\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665506 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665496    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-data\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665512    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-podinfo\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.665601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.665541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-socket\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766528 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-token\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766528 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-credential-socket\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766762 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istiod-ca-cert\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766762 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-envoy\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766853 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-certs\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766853 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-data\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766853 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-podinfo\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766995 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-socket\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766995 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pblk9\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-kube-api-access-pblk9\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.766995 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.766971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-credential-socket\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.767188 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.767168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-data\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.767257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.767233    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-certs\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.767458 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.767428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-socket\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.767560 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.767471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istiod-ca-cert\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.769019 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.768994    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-envoy\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.769305 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.769282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-podinfo\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.775335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.775307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-token\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.775569 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.775553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pblk9\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-kube-api-access-pblk9\") pod \"e2e-api-b-8c14db-openshift-default-645665cdcc-946d5\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") " pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.931050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.930951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:05.990103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:05.990072    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:05:06.071958 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.071932    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"]
Jul 16 23:05:06.073643 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:05:06.073614    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450dc9a9_e95f_4b5d_b510_eebc3fdc98aa.slice/crio-1ad5df41a357e83fe96ac90df93ded63e653aa68f646c7a8086df5e4b6d28d19 WatchSource:0}: Error finding container 1ad5df41a357e83fe96ac90df93ded63e653aa68f646c7a8086df5e4b6d28d19: Status 404 returned error can't find the container with id 1ad5df41a357e83fe96ac90df93ded63e653aa68f646c7a8086df5e4b6d28d19
Jul 16 23:05:06.075703 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.075671    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:06.075808 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.075751    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:06.075808 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.075794    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:06.530399 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.530349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" event={"ID":"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa","Type":"ContainerStarted","Data":"ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"}
Jul 16 23:05:06.530599 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.530441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" event={"ID":"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa","Type":"ContainerStarted","Data":"1ad5df41a357e83fe96ac90df93ded63e653aa68f646c7a8086df5e4b6d28d19"}
Jul 16 23:05:06.550204 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.550129    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" podStartSLOduration=1.550111695 podStartE2EDuration="1.550111695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:05:06.548534312 +0000 UTC m=+2862.869090418" watchObservedRunningTime="2026-07-16 23:05:06.550111695 +0000 UTC m=+2862.870667767"
Jul 16 23:05:06.932031 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:06.931953    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:07.936799 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:07.936769    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:08.542811 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:08.542777    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:08.544014 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:08.543993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:14.595222 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:14.595187    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerID="7b363dccda85ab754d55386cf7be3d3d0aa49bf366e7b826fcc118c1cd90c7dd" exitCode=0
Jul 16 23:05:14.595671 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:14.595261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" event={"ID":"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f","Type":"ContainerDied","Data":"7b363dccda85ab754d55386cf7be3d3d0aa49bf366e7b826fcc118c1cd90c7dd"}
Jul 16 23:05:14.595733 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:14.595682    2564 scope.go:117] "RemoveContainer" containerID="7b363dccda85ab754d55386cf7be3d3d0aa49bf366e7b826fcc118c1cd90c7dd"
Jul 16 23:05:14.596775 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:14.596744    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:14.596848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:14.596829    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:14.596912 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:14.596895    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:15.495302 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:15.495213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:15.603804 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:15.603770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" event={"ID":"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f","Type":"ContainerStarted","Data":"0bdc0bbe33c633a8bae7f443465315fb413e846f5e195862f91fc545713c485e"}
Jul 16 23:05:15.820941 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:15.820850    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:15.822061 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:15.822037    2564 patch_prober.go:28] interesting pod/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Jul 16 23:05:15.822120 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:15.822103    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Jul 16 23:05:16.820951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:16.820916    2564 patch_prober.go:28] interesting pod/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Jul 16 23:05:16.821397 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:16.820970    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Jul 16 23:05:17.821502 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:17.821462    2564 patch_prober.go:28] interesting pod/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Jul 16 23:05:17.821998 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:17.821530    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Jul 16 23:05:18.821228 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:18.821188    2564 patch_prober.go:28] interesting pod/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Jul 16 23:05:18.821436 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:18.821248    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Jul 16 23:05:19.679212 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:19.679172    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:05:19.820885 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:19.820840    2564 patch_prober.go:28] interesting pod/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Jul 16 23:05:19.821056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:19.820916    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Jul 16 23:05:20.638170 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.638066    2564 generic.go:356] "Generic (PLEG): container finished" podID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerID="ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7" exitCode=0
Jul 16 23:05:20.638170 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.638156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" event={"ID":"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa","Type":"ContainerDied","Data":"ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"}
Jul 16 23:05:20.638550 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.638536    2564 scope.go:117] "RemoveContainer" containerID="ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"
Jul 16 23:05:20.639505 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.639482    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:20.639560 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.639548    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:20.639600 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.639587    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:05:20.825519 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.825487    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:20.825916 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.825825    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:20.826539 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:20.826517    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:21.646469 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:21.646424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" event={"ID":"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa","Type":"ContainerStarted","Data":"828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943"}
Jul 16 23:05:21.931556 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:21.931462    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:21.932655 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:21.932623    2564 patch_prober.go:28] interesting pod/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Jul 16 23:05:21.932779 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:21.932678    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Jul 16 23:05:22.546676 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.546639    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"]
Jul 16 23:05:22.547063 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.547050    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9cfefab-0cb2-488b-af0e-def2b67261b5" containerName="istio-proxy"
Jul 16 23:05:22.549041 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.549024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:22.551278 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.551255    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:05:22.551278 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.551262    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:05:22.557767 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.557706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"]
Jul 16 23:05:22.636899 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.636862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72a09097-4991-47aa-bf06-d97683d3647e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg\" (UID: \"72a09097-4991-47aa-bf06-d97683d3647e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:22.738285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.738233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72a09097-4991-47aa-bf06-d97683d3647e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg\" (UID: \"72a09097-4991-47aa-bf06-d97683d3647e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:22.738874 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.738853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72a09097-4991-47aa-bf06-d97683d3647e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg\" (UID: \"72a09097-4991-47aa-bf06-d97683d3647e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:22.861024 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.860931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:22.931750 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.931705    2564 patch_prober.go:28] interesting pod/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Jul 16 23:05:22.932119 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:22.931775    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Jul 16 23:05:23.044904 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.044879    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"]
Jul 16 23:05:23.046689 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:05:23.046661    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a09097_4991_47aa_bf06_d97683d3647e.slice/crio-d58dd1fdc94921e84f28ff564c89b6b4c3aa5c5a40dacaa2e412ef88949851ca WatchSource:0}: Error finding container d58dd1fdc94921e84f28ff564c89b6b4c3aa5c5a40dacaa2e412ef88949851ca: Status 404 returned error can't find the container with id d58dd1fdc94921e84f28ff564c89b6b4c3aa5c5a40dacaa2e412ef88949851ca
Jul 16 23:05:23.543676 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.543591    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:23.664584 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.664549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" event={"ID":"72a09097-4991-47aa-bf06-d97683d3647e","Type":"ContainerStarted","Data":"2cacc9805d9b256afdf1bf2b2373c272da579f72d88bc4b4d1f974fa212f6290"}
Jul 16 23:05:23.664792 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.664597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" event={"ID":"72a09097-4991-47aa-bf06-d97683d3647e","Type":"ContainerStarted","Data":"d58dd1fdc94921e84f28ff564c89b6b4c3aa5c5a40dacaa2e412ef88949851ca"}
Jul 16 23:05:23.682335 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.682264    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" podStartSLOduration=1.682240738 podStartE2EDuration="1.682240738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:05:23.679799092 +0000 UTC m=+2880.000355165" watchObservedRunningTime="2026-07-16 23:05:23.682240738 +0000 UTC m=+2880.002796811"
Jul 16 23:05:23.932396 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.932307    2564 patch_prober.go:28] interesting pod/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Jul 16 23:05:23.932396 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:23.932379    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Jul 16 23:05:24.673650 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:24.673604    2564 generic.go:356] "Generic (PLEG): container finished" podID="72a09097-4991-47aa-bf06-d97683d3647e" containerID="2cacc9805d9b256afdf1bf2b2373c272da579f72d88bc4b4d1f974fa212f6290" exitCode=6
Jul 16 23:05:24.673924 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:24.673681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" event={"ID":"72a09097-4991-47aa-bf06-d97683d3647e","Type":"ContainerDied","Data":"2cacc9805d9b256afdf1bf2b2373c272da579f72d88bc4b4d1f974fa212f6290"}
Jul 16 23:05:24.674189 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:24.674119    2564 scope.go:117] "RemoveContainer" containerID="2cacc9805d9b256afdf1bf2b2373c272da579f72d88bc4b4d1f974fa212f6290"
Jul 16 23:05:24.931882 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:24.931794    2564 patch_prober.go:28] interesting pod/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Jul 16 23:05:24.931882 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:24.931852    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Jul 16 23:05:25.682271 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:25.682233    2564 generic.go:356] "Generic (PLEG): container finished" podID="72a09097-4991-47aa-bf06-d97683d3647e" containerID="ff34168056d1efbe55c7eed059d87e0db1325271fb3246288289eea653b26dac" exitCode=6
Jul 16 23:05:25.682698 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:25.682353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" event={"ID":"72a09097-4991-47aa-bf06-d97683d3647e","Type":"ContainerDied","Data":"ff34168056d1efbe55c7eed059d87e0db1325271fb3246288289eea653b26dac"}
Jul 16 23:05:25.682698 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:25.682419    2564 scope.go:117] "RemoveContainer" containerID="2cacc9805d9b256afdf1bf2b2373c272da579f72d88bc4b4d1f974fa212f6290"
Jul 16 23:05:25.682698 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:25.682649    2564 scope.go:117] "RemoveContainer" containerID="ff34168056d1efbe55c7eed059d87e0db1325271fb3246288289eea653b26dac"
Jul 16 23:05:25.682924 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:05:25.682903    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg_odh-ai-gateway-infra(72a09097-4991-47aa-bf06-d97683d3647e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" podUID="72a09097-4991-47aa-bf06-d97683d3647e"
Jul 16 23:05:25.935459 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:25.935380    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:25.936630 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:25.936606    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:05:26.689383 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:26.689347    2564 scope.go:117] "RemoveContainer" containerID="ff34168056d1efbe55c7eed059d87e0db1325271fb3246288289eea653b26dac"
Jul 16 23:05:26.689761 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:05:26.689547    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg_odh-ai-gateway-infra(72a09097-4991-47aa-bf06-d97683d3647e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" podUID="72a09097-4991-47aa-bf06-d97683d3647e"
Jul 16 23:05:35.480650 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:35.480616    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:05:37.596697 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:37.596660    2564 scope.go:117] "RemoveContainer" containerID="ff34168056d1efbe55c7eed059d87e0db1325271fb3246288289eea653b26dac"
Jul 16 23:05:38.757613 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:38.757568    2564 generic.go:356] "Generic (PLEG): container finished" podID="72a09097-4991-47aa-bf06-d97683d3647e" containerID="6a587e94d4e0bd549e5d82ee6888ea1cb22ffd9df0cafcb49de8f022f68645c5" exitCode=6
Jul 16 23:05:38.758102 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:38.757626    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" event={"ID":"72a09097-4991-47aa-bf06-d97683d3647e","Type":"ContainerDied","Data":"6a587e94d4e0bd549e5d82ee6888ea1cb22ffd9df0cafcb49de8f022f68645c5"}
Jul 16 23:05:38.758102 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:38.757688    2564 scope.go:117] "RemoveContainer" containerID="ff34168056d1efbe55c7eed059d87e0db1325271fb3246288289eea653b26dac"
Jul 16 23:05:38.758102 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:38.757948    2564 scope.go:117] "RemoveContainer" containerID="6a587e94d4e0bd549e5d82ee6888ea1cb22ffd9df0cafcb49de8f022f68645c5"
Jul 16 23:05:38.758312 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:05:38.758204    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg_odh-ai-gateway-infra(72a09097-4991-47aa-bf06-d97683d3647e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" podUID="72a09097-4991-47aa-bf06-d97683d3647e"
Jul 16 23:05:39.784644 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:39.784609    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"]
Jul 16 23:05:39.939288 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:39.939260    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:40.102994 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.102905    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72a09097-4991-47aa-bf06-d97683d3647e-maas-api-service-ca\") pod \"72a09097-4991-47aa-bf06-d97683d3647e\" (UID: \"72a09097-4991-47aa-bf06-d97683d3647e\") "
Jul 16 23:05:40.103345 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.103320    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72a09097-4991-47aa-bf06-d97683d3647e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72a09097-4991-47aa-bf06-d97683d3647e" (UID: "72a09097-4991-47aa-bf06-d97683d3647e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:05:40.203997 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.203958    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72a09097-4991-47aa-bf06-d97683d3647e-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:40.771985 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.771953    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"
Jul 16 23:05:40.771985 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.771965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg" event={"ID":"72a09097-4991-47aa-bf06-d97683d3647e","Type":"ContainerDied","Data":"d58dd1fdc94921e84f28ff564c89b6b4c3aa5c5a40dacaa2e412ef88949851ca"}
Jul 16 23:05:40.772254 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.772027    2564 scope.go:117] "RemoveContainer" containerID="6a587e94d4e0bd549e5d82ee6888ea1cb22ffd9df0cafcb49de8f022f68645c5"
Jul 16 23:05:40.798100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.798064    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"]
Jul 16 23:05:40.804225 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:40.804194    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-94hlg"]
Jul 16 23:05:41.601631 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:41.601597    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a09097-4991-47aa-bf06-d97683d3647e" path="/var/lib/kubelet/pods/72a09097-4991-47aa-bf06-d97683d3647e/volumes"
Jul 16 23:05:46.622338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:46.622305    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"]
Jul 16 23:05:46.622771 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:46.622493    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" containerID="cri-o://0bdc0bbe33c633a8bae7f443465315fb413e846f5e195862f91fc545713c485e" gracePeriod=30
Jul 16 23:05:50.828063 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:50.828025    2564 patch_prober.go:28] interesting pod/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 23:05:50.828470 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:50.828099    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 23:05:51.837933 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.837889    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerID="0bdc0bbe33c633a8bae7f443465315fb413e846f5e195862f91fc545713c485e" exitCode=0
Jul 16 23:05:51.838349 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.837954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" event={"ID":"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f","Type":"ContainerDied","Data":"0bdc0bbe33c633a8bae7f443465315fb413e846f5e195862f91fc545713c485e"}
Jul 16 23:05:51.838349 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.838012    2564 scope.go:117] "RemoveContainer" containerID="7b363dccda85ab754d55386cf7be3d3d0aa49bf366e7b826fcc118c1cd90c7dd"
Jul 16 23:05:51.878564 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.878540    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:51.904686 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.904598    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spkrb\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-kube-api-access-spkrb\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.904846 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.904695    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istiod-ca-cert\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.904846 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.904723    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-certs\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.904846 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.904770    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-token\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.905078 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905038    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-socket\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.905155 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905102    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-podinfo\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.905225 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905190    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:51.905225 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-credential-socket\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.905330 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905229    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-envoy\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.905925 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905710    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-data\") pod \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\" (UID: \"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f\") "
Jul 16 23:05:51.905925 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905713    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:51.905925 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905731    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:51.905925 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.905893    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:05:51.906249 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.906217    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:51.906249 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.906237    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:51.906353 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.906250    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:51.906353 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.906265    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:51.906752 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.906716    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:51.907473 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.907436    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-kube-api-access-spkrb" (OuterVolumeSpecName: "kube-api-access-spkrb") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "kube-api-access-spkrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:05:51.907822 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.907800    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:05:51.908325 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.908302    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:05:51.908325 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:51.908316    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" (UID: "7f8b4cbf-103b-4d45-97bc-af724a7e9f4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:05:52.006717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.006680    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:52.006717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.006711    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:52.006717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.006725    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spkrb\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-kube-api-access-spkrb\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:52.006971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.006747    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:52.006971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.006759    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:05:52.844768 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.844739    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv" event={"ID":"7f8b4cbf-103b-4d45-97bc-af724a7e9f4f","Type":"ContainerDied","Data":"28dc39864d02268b1705fbcb3aca8aec2cab69d45d6be773c9686a01e93c7f70"}
Jul 16 23:05:52.844768 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.844767    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"
Jul 16 23:05:52.845288 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.844795    2564 scope.go:117] "RemoveContainer" containerID="0bdc0bbe33c633a8bae7f443465315fb413e846f5e195862f91fc545713c485e"
Jul 16 23:05:52.871238 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.871201    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"]
Jul 16 23:05:52.873455 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:52.873431    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b9c909-openshift-default-55c65c7499-v4lnv"]
Jul 16 23:05:53.602275 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:05:53.602241    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" path="/var/lib/kubelet/pods/7f8b4cbf-103b-4d45-97bc-af724a7e9f4f/volumes"
Jul 16 23:06:00.785637 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:00.785602    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:06:12.673582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:12.673549    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"]
Jul 16 23:06:12.674064 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:12.673737    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy" containerID="cri-o://828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943" gracePeriod=30
Jul 16 23:06:13.567637 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.567599    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"]
Jul 16 23:06:13.568066 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568047    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568068    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568079    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568087    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568100    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568109    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568120    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy"
Jul 16 23:06:13.568195 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568128    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy"
Jul 16 23:06:13.568548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568260    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:13.568548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568277    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy"
Jul 16 23:06:13.568548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568291    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8b4cbf-103b-4d45-97bc-af724a7e9f4f" containerName="istio-proxy"
Jul 16 23:06:13.568548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.568300    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:13.571437 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.571417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.574011 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.573990    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b5738f-openshift-default-dockercfg-5pvfr\""
Jul 16 23:06:13.584087 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.584062    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"]
Jul 16 23:06:13.698010 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.697973    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-credential-socket\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698010 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698010    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istiod-ca-cert\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-socket\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698059    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-podinfo\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvsj\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-kube-api-access-kzvsj\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-envoy\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-certs\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-token\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.698509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.698253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-data\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.798988 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.798953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-envoy\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799202 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-certs\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799202 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799028    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-token\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799202 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-data\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799202 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-credential-socket\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799457 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istiod-ca-cert\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799598 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-credential-socket\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799598 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-socket\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799598 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-certs\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799598 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-podinfo\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799598 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-data\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799903 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799640    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvsj\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-kube-api-access-kzvsj\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.799903 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.799706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-socket\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.800104 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.800086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istiod-ca-cert\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.801275 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.801258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-envoy\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.801887 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.801871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-podinfo\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.808183 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.808148    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-token\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.808307 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.808204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvsj\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-kube-api-access-kzvsj\") pod \"e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") " pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:13.882897 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:13.882806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:14.020442 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.020417    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"]
Jul 16 23:06:14.022663 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:06:14.022635    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee519fb8_ae9d_47a2_b2ae_9b8cece5951c.slice/crio-37c02af40e4fc101becd53d68584aee0dc68a45c6abf9a12b814a99db84b833c WatchSource:0}: Error finding container 37c02af40e4fc101becd53d68584aee0dc68a45c6abf9a12b814a99db84b833c: Status 404 returned error can't find the container with id 37c02af40e4fc101becd53d68584aee0dc68a45c6abf9a12b814a99db84b833c
Jul 16 23:06:14.024753 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.024720    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:14.024843 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.024785    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:14.024843 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.024814    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:14.971719 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.971675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" event={"ID":"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c","Type":"ContainerStarted","Data":"8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"}
Jul 16 23:06:14.972121 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.971738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" event={"ID":"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c","Type":"ContainerStarted","Data":"37c02af40e4fc101becd53d68584aee0dc68a45c6abf9a12b814a99db84b833c"}
Jul 16 23:06:14.993721 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:14.993664    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" podStartSLOduration=1.9936470929999999 podStartE2EDuration="1.993647093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:06:14.990804729 +0000 UTC m=+2931.311360801" watchObservedRunningTime="2026-07-16 23:06:14.993647093 +0000 UTC m=+2931.314203167"
Jul 16 23:06:15.883210 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:15.883172    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:15.888295 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:15.888262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:15.977617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:15.977585    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:15.978806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:15.978788    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:17.935948 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.935881    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:06:17.989551 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.989514    2564 generic.go:356] "Generic (PLEG): container finished" podID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerID="828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943" exitCode=0
Jul 16 23:06:17.989755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.989560    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"
Jul 16 23:06:17.989755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.989593    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" event={"ID":"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa","Type":"ContainerDied","Data":"828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943"}
Jul 16 23:06:17.989755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.989661    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5" event={"ID":"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa","Type":"ContainerDied","Data":"1ad5df41a357e83fe96ac90df93ded63e653aa68f646c7a8086df5e4b6d28d19"}
Jul 16 23:06:17.989755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.989694    2564 scope.go:117] "RemoveContainer" containerID="828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943"
Jul 16 23:06:17.998721 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:17.998703    2564 scope.go:117] "RemoveContainer" containerID="ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"
Jul 16 23:06:18.006940 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.006921    2564 scope.go:117] "RemoveContainer" containerID="828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943"
Jul 16 23:06:18.007193 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:06:18.007172    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943\": container with ID starting with 828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943 not found: ID does not exist" containerID="828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943"
Jul 16 23:06:18.007244 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.007202    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943"} err="failed to get container status \"828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943\": rpc error: code = NotFound desc = could not find container \"828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943\": container with ID starting with 828fcda75892b6170ae80a0d175db0d8dc6115d20617be9cd988149375166943 not found: ID does not exist"
Jul 16 23:06:18.007244 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.007222    2564 scope.go:117] "RemoveContainer" containerID="ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"
Jul 16 23:06:18.007463 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:06:18.007448    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7\": container with ID starting with ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7 not found: ID does not exist" containerID="ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"
Jul 16 23:06:18.007503 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.007468    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7"} err="failed to get container status \"ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7\": rpc error: code = NotFound desc = could not find container \"ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7\": container with ID starting with ceb6c1e7efbfbfb0a044aea434bf27eda311e3326747556a0bf7559b04ed93e7 not found: ID does not exist"
Jul 16 23:06:18.046031 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.045997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-credential-socket\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046031 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046032    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istiod-ca-cert\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046269 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046050    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-certs\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046269 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046084    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-data\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046269 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-podinfo\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046269 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046219    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-token\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046269 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046246    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-socket\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046529 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046272    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pblk9\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-kube-api-access-pblk9\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046529 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046302    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:06:18.046529 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-envoy\") pod \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\" (UID: \"450dc9a9-e95f-4b5d-b510-eebc3fdc98aa\") "
Jul 16 23:06:18.046529 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046401    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:06:18.046758 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046552    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:06:18.046758 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046616    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:06:18.046869 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046835    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.046869 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046856    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.046978 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046869    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.046978 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.046883    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.047367 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.047332    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:06:18.048561 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.048524    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:06:18.048561 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.048532    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:06:18.048699 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.048575    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-kube-api-access-pblk9" (OuterVolumeSpecName: "kube-api-access-pblk9") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "kube-api-access-pblk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:06:18.048699 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.048599    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" (UID: "450dc9a9-e95f-4b5d-b510-eebc3fdc98aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:06:18.148105 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.148072    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.148105 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.148106    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.148105 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.148118    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.148484 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.148158    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pblk9\" (UniqueName: \"kubernetes.io/projected/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-kube-api-access-pblk9\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.148484 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.148174    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:18.312684 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.312647    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"]
Jul 16 23:06:18.316777 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:18.316751    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8c14db-openshift-default-645665cdcc-946d5"]
Jul 16 23:06:19.169772 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.169734    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"]
Jul 16 23:06:19.170152 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.170062    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy"
Jul 16 23:06:19.170152 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.170072    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy"
Jul 16 23:06:19.170152 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.170095    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:19.170152 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.170100    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:19.170304 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.170179    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a09097-4991-47aa-bf06-d97683d3647e" containerName="revoke-keys"
Jul 16 23:06:19.170304 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.170193    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy"
Jul 16 23:06:19.174701 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.174680    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:19.178538 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.178511    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:06:19.178717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.178561    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:06:19.181236 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.181211    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"]
Jul 16 23:06:19.259612 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.259577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f18b775-1292-4b7a-ada2-0a077e0f29dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-48cl9\" (UID: \"7f18b775-1292-4b7a-ada2-0a077e0f29dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:19.360264 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.360218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f18b775-1292-4b7a-ada2-0a077e0f29dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-48cl9\" (UID: \"7f18b775-1292-4b7a-ada2-0a077e0f29dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:19.360871 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.360848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f18b775-1292-4b7a-ada2-0a077e0f29dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-48cl9\" (UID: \"7f18b775-1292-4b7a-ada2-0a077e0f29dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:19.487390 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.487350    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:19.601960 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.601929    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" path="/var/lib/kubelet/pods/450dc9a9-e95f-4b5d-b510-eebc3fdc98aa/volumes"
Jul 16 23:06:19.626287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:19.626242    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"]
Jul 16 23:06:19.628606 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:06:19.628578    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f18b775_1292_4b7a_ada2_0a077e0f29dc.slice/crio-79b97ac9314d42c624a911ba40653a8c08fb2ccd532c2c2d6b58739600be38a0 WatchSource:0}: Error finding container 79b97ac9314d42c624a911ba40653a8c08fb2ccd532c2c2d6b58739600be38a0: Status 404 returned error can't find the container with id 79b97ac9314d42c624a911ba40653a8c08fb2ccd532c2c2d6b58739600be38a0
Jul 16 23:06:20.004892 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:20.004856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerStarted","Data":"ffda7ae2f6b8201e11260847dfa9c55ce868f54d25f5cd412326b0157373e75a"}
Jul 16 23:06:20.005091 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:20.004956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerStarted","Data":"79b97ac9314d42c624a911ba40653a8c08fb2ccd532c2c2d6b58739600be38a0"}
Jul 16 23:06:20.022301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:20.022245    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" podStartSLOduration=1.022229563 podStartE2EDuration="1.022229563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:06:20.019960653 +0000 UTC m=+2936.340516736" watchObservedRunningTime="2026-07-16 23:06:20.022229563 +0000 UTC m=+2936.342785673"
Jul 16 23:06:21.011605 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.011573    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerID="ffda7ae2f6b8201e11260847dfa9c55ce868f54d25f5cd412326b0157373e75a" exitCode=6
Jul 16 23:06:21.012029 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.011616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerDied","Data":"ffda7ae2f6b8201e11260847dfa9c55ce868f54d25f5cd412326b0157373e75a"}
Jul 16 23:06:21.012029 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.011814    2564 scope.go:117] "RemoveContainer" containerID="ffda7ae2f6b8201e11260847dfa9c55ce868f54d25f5cd412326b0157373e75a"
Jul 16 23:06:21.780545 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.780509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:06:21.966336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.966307    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"]
Jul 16 23:06:21.966688 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.966677    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy"
Jul 16 23:06:21.966741 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.966689    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy"
Jul 16 23:06:21.966778 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.966757    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="450dc9a9-e95f-4b5d-b510-eebc3fdc98aa" containerName="istio-proxy"
Jul 16 23:06:21.969872 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.969855    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:21.972320 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.972299    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-61f04e-openshift-default-dockercfg-rpsbh\""
Jul 16 23:06:21.985205 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:21.985174    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"]
Jul 16 23:06:22.018974 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.018936    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerID="32b1f2b08b47f4f614fdeb61bad5723c6c9217015b27303bfc652d78a519de8c" exitCode=6
Jul 16 23:06:22.019405 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.019030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerDied","Data":"32b1f2b08b47f4f614fdeb61bad5723c6c9217015b27303bfc652d78a519de8c"}
Jul 16 23:06:22.019405 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.019083    2564 scope.go:117] "RemoveContainer" containerID="ffda7ae2f6b8201e11260847dfa9c55ce868f54d25f5cd412326b0157373e75a"
Jul 16 23:06:22.019405 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.019234    2564 scope.go:117] "RemoveContainer" containerID="32b1f2b08b47f4f614fdeb61bad5723c6c9217015b27303bfc652d78a519de8c"
Jul 16 23:06:22.019581 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:06:22.019485    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-48cl9_odh-ai-gateway-infra(7f18b775-1292-4b7a-ada2-0a077e0f29dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc"
Jul 16 23:06:22.084835 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.084804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-credential-socket\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085011 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.084843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-token\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085011 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.084864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-socket\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085011 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.084922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftwxw\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-kube-api-access-ftwxw\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085011 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.084986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-data\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085174 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.085012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435bef17-4ce1-484a-976d-c117d10e9ddb-istiod-ca-cert\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085174 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.085076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-envoy\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085174 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.085110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-podinfo\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.085174 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.085152    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-certs\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186382 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-credential-socket\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-token\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-socket\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftwxw\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-kube-api-access-ftwxw\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-data\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435bef17-4ce1-484a-976d-c117d10e9ddb-istiod-ca-cert\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-envoy\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186933 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-podinfo\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186933 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-certs\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186933 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186803    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-credential-socket\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.186933 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.186862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-socket\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.187130 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.187036    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-data\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.187130 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.187102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-certs\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.187299 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.187280    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435bef17-4ce1-484a-976d-c117d10e9ddb-istiod-ca-cert\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.188917 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.188885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-envoy\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.189169 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.189150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-podinfo\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.193993 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.193973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-token\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.194254 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.194238    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftwxw\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-kube-api-access-ftwxw\") pod \"e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") " pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.280662 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.280563    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:22.425821 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.425793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"]
Jul 16 23:06:22.426718 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:06:22.426689    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435bef17_4ce1_484a_976d_c117d10e9ddb.slice/crio-aef86aa1db82fca0e9aae9185fa52188ba510a6b336fe33d9781c6b57659ad3f WatchSource:0}: Error finding container aef86aa1db82fca0e9aae9185fa52188ba510a6b336fe33d9781c6b57659ad3f: Status 404 returned error can't find the container with id aef86aa1db82fca0e9aae9185fa52188ba510a6b336fe33d9781c6b57659ad3f
Jul 16 23:06:22.428843 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.428811    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:22.428938 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.428878    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:22.428938 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:22.428905    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:23.027317 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:23.027279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" event={"ID":"435bef17-4ce1-484a-976d-c117d10e9ddb","Type":"ContainerStarted","Data":"40418b7d521a9ab61956b0e976a31aedfffbe6521d4f22efb0ac3ff536a7ac02"}
Jul 16 23:06:23.027806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:23.027338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" event={"ID":"435bef17-4ce1-484a-976d-c117d10e9ddb","Type":"ContainerStarted","Data":"aef86aa1db82fca0e9aae9185fa52188ba510a6b336fe33d9781c6b57659ad3f"}
Jul 16 23:06:23.029017 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:23.028997    2564 scope.go:117] "RemoveContainer" containerID="32b1f2b08b47f4f614fdeb61bad5723c6c9217015b27303bfc652d78a519de8c"
Jul 16 23:06:23.029239 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:06:23.029219    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-48cl9_odh-ai-gateway-infra(7f18b775-1292-4b7a-ada2-0a077e0f29dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc"
Jul 16 23:06:23.049791 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:23.049733    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podStartSLOduration=2.049716661 podStartE2EDuration="2.049716661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:06:23.046792755 +0000 UTC m=+2939.367348828" watchObservedRunningTime="2026-07-16 23:06:23.049716661 +0000 UTC m=+2939.370272729"
Jul 16 23:06:23.281926 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:23.281821    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:23.294657 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:23.294619    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:24.034243 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:24.034207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:24.035235 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:24.035211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:25.001550 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:25.001513    2564 scope.go:117] "RemoveContainer" containerID="3aa884d82e987a3ec2a67f5c4162f7402b6a81f9f1b60b7de1b5332e2f6aec94"
Jul 16 23:06:30.067345 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.067312    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerID="8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0" exitCode=0
Jul 16 23:06:30.067783 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.067365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" event={"ID":"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c","Type":"ContainerDied","Data":"8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"}
Jul 16 23:06:30.067783 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.067719    2564 scope.go:117] "RemoveContainer" containerID="8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"
Jul 16 23:06:30.068710 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.068685    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:30.068783 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.068748    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:30.068783 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.068776    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:30.984159 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:30.979249    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:31.079732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:31.079688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" event={"ID":"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c","Type":"ContainerStarted","Data":"2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4"}
Jul 16 23:06:31.883768 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:31.883735    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:31.885367 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:31.885336    2564 patch_prober.go:28] interesting pod/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Jul 16 23:06:31.885578 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:31.885551    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Jul 16 23:06:32.884537 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:32.884497    2564 patch_prober.go:28] interesting pod/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Jul 16 23:06:32.885164 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:32.885110    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Jul 16 23:06:33.393159 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:33.393089    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:06:33.884080 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:33.884040    2564 patch_prober.go:28] interesting pod/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Jul 16 23:06:33.884305 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:33.884121    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Jul 16 23:06:34.884038 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:34.883993    2564 patch_prober.go:28] interesting pod/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Jul 16 23:06:34.885201 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:34.884677    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Jul 16 23:06:35.890675 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:35.890642    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:35.891680 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:35.891651    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:35.893644 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:35.893619    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:06:37.597002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:37.596966    2564 scope.go:117] "RemoveContainer" containerID="32b1f2b08b47f4f614fdeb61bad5723c6c9217015b27303bfc652d78a519de8c"
Jul 16 23:06:38.126922 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:38.126884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerStarted","Data":"1d93ec0776fd8746f4f6d6b23c8f74d28ba682ae4fbf1575c3e8e168cc42971e"}
Jul 16 23:06:39.134419 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:39.134381    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerID="1d93ec0776fd8746f4f6d6b23c8f74d28ba682ae4fbf1575c3e8e168cc42971e" exitCode=6
Jul 16 23:06:39.134928 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:39.134457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerDied","Data":"1d93ec0776fd8746f4f6d6b23c8f74d28ba682ae4fbf1575c3e8e168cc42971e"}
Jul 16 23:06:39.134928 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:39.134520    2564 scope.go:117] "RemoveContainer" containerID="32b1f2b08b47f4f614fdeb61bad5723c6c9217015b27303bfc652d78a519de8c"
Jul 16 23:06:39.134928 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:39.134786    2564 scope.go:117] "RemoveContainer" containerID="1d93ec0776fd8746f4f6d6b23c8f74d28ba682ae4fbf1575c3e8e168cc42971e"
Jul 16 23:06:39.135100 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:06:39.135041    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-48cl9_odh-ai-gateway-infra(7f18b775-1292-4b7a-ada2-0a077e0f29dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc"
Jul 16 23:06:39.164858 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:39.164825    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"]
Jul 16 23:06:40.076323 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:40.076280    2564 patch_prober.go:28] interesting pod/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 23:06:40.076513 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:40.076351    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 23:06:40.281629 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:40.281598    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:40.367342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:40.367263    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f18b775-1292-4b7a-ada2-0a077e0f29dc-maas-api-service-ca\") pod \"7f18b775-1292-4b7a-ada2-0a077e0f29dc\" (UID: \"7f18b775-1292-4b7a-ada2-0a077e0f29dc\") "
Jul 16 23:06:40.367617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:40.367595    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f18b775-1292-4b7a-ada2-0a077e0f29dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f18b775-1292-4b7a-ada2-0a077e0f29dc" (UID: "7f18b775-1292-4b7a-ada2-0a077e0f29dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:06:40.468179 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:40.468118    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f18b775-1292-4b7a-ada2-0a077e0f29dc-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:06:41.149604 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:41.149569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9" event={"ID":"7f18b775-1292-4b7a-ada2-0a077e0f29dc","Type":"ContainerDied","Data":"79b97ac9314d42c624a911ba40653a8c08fb2ccd532c2c2d6b58739600be38a0"}
Jul 16 23:06:41.150706 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:41.149637    2564 scope.go:117] "RemoveContainer" containerID="1d93ec0776fd8746f4f6d6b23c8f74d28ba682ae4fbf1575c3e8e168cc42971e"
Jul 16 23:06:41.150706 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:41.149579    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"
Jul 16 23:06:41.174342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:41.174314    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"]
Jul 16 23:06:41.179652 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:41.179617    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-48cl9"]
Jul 16 23:06:41.601661 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:41.601623    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" path="/var/lib/kubelet/pods/7f18b775-1292-4b7a-ada2-0a077e0f29dc/volumes"
Jul 16 23:06:42.167092 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:42.167053    2564 generic.go:356] "Generic (PLEG): container finished" podID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerID="40418b7d521a9ab61956b0e976a31aedfffbe6521d4f22efb0ac3ff536a7ac02" exitCode=0
Jul 16 23:06:42.167301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:42.167122    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" event={"ID":"435bef17-4ce1-484a-976d-c117d10e9ddb","Type":"ContainerDied","Data":"40418b7d521a9ab61956b0e976a31aedfffbe6521d4f22efb0ac3ff536a7ac02"}
Jul 16 23:06:42.167618 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:42.167594    2564 scope.go:117] "RemoveContainer" containerID="40418b7d521a9ab61956b0e976a31aedfffbe6521d4f22efb0ac3ff536a7ac02"
Jul 16 23:06:42.168848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:42.168818    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:42.168918 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:42.168905    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:42.168969 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:42.168953    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 23:06:43.179313 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:43.179268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" event={"ID":"435bef17-4ce1-484a-976d-c117d10e9ddb","Type":"ContainerStarted","Data":"344f553850f93883b138492ca6fee3b037f636a0c0abc4c8330966201bdda743"}
Jul 16 23:06:43.281457 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:43.281414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:43.282235 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:43.281889    2564 patch_prober.go:28] interesting pod/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Jul 16 23:06:43.282235 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:43.281943    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Jul 16 23:06:44.281210 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:44.281159    2564 patch_prober.go:28] interesting pod/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Jul 16 23:06:44.281809 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:44.281232    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Jul 16 23:06:45.281064 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:45.281021    2564 patch_prober.go:28] interesting pod/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Jul 16 23:06:45.281414 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:45.281379    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Jul 16 23:06:46.281461 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:46.281418    2564 patch_prober.go:28] interesting pod/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Jul 16 23:06:46.281964 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:46.281490    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Jul 16 23:06:46.495283 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:46.495242    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:06:47.285351 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:47.285317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:47.285738 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:47.285574    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:06:47.286351 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:06:47.286333    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:07:02.640845 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.640806    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"]
Jul 16 23:07:02.641285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641168    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.641285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641183    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.641285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641200    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.641285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641206    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.641426 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641296    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.641426 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641305    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.641426 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.641311    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:02.643902 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.643884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:02.646218 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.646193    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:07:02.646218 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.646209    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:07:02.652473 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.652451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"]
Jul 16 23:07:02.766287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.766241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/542f46da-4e8a-48dc-acb4-b55405a05dd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq\" (UID: \"542f46da-4e8a-48dc-acb4-b55405a05dd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:02.867543 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.867513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/542f46da-4e8a-48dc-acb4-b55405a05dd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq\" (UID: \"542f46da-4e8a-48dc-acb4-b55405a05dd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:02.868114 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.868095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/542f46da-4e8a-48dc-acb4-b55405a05dd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq\" (UID: \"542f46da-4e8a-48dc-acb4-b55405a05dd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:02.955993 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:02.955965    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:03.089302 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:03.089277    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"]
Jul 16 23:07:03.090803 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:07:03.090766    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542f46da_4e8a_48dc_acb4_b55405a05dd4.slice/crio-5c19e893498eb56846e99b552bf9b987032a4c1cc29ec2bae59dac2e114ba9ad WatchSource:0}: Error finding container 5c19e893498eb56846e99b552bf9b987032a4c1cc29ec2bae59dac2e114ba9ad: Status 404 returned error can't find the container with id 5c19e893498eb56846e99b552bf9b987032a4c1cc29ec2bae59dac2e114ba9ad
Jul 16 23:07:03.299049 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:03.299015    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" event={"ID":"542f46da-4e8a-48dc-acb4-b55405a05dd4","Type":"ContainerStarted","Data":"5c19e893498eb56846e99b552bf9b987032a4c1cc29ec2bae59dac2e114ba9ad"}
Jul 16 23:07:04.306546 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:04.306511    2564 generic.go:356] "Generic (PLEG): container finished" podID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerID="e87ccc7ce5ebad384320a1c3b4791f40b97f7ab77eab41ed4b43651929b6f6e0" exitCode=6
Jul 16 23:07:04.306959 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:04.306559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" event={"ID":"542f46da-4e8a-48dc-acb4-b55405a05dd4","Type":"ContainerDied","Data":"e87ccc7ce5ebad384320a1c3b4791f40b97f7ab77eab41ed4b43651929b6f6e0"}
Jul 16 23:07:04.306959 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:04.306803    2564 scope.go:117] "RemoveContainer" containerID="e87ccc7ce5ebad384320a1c3b4791f40b97f7ab77eab41ed4b43651929b6f6e0"
Jul 16 23:07:05.313802 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:05.313714    2564 generic.go:356] "Generic (PLEG): container finished" podID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerID="77c2e021230f6a050a91df8df95c34eedbc84fbd0c51f689a34e2df5424978c1" exitCode=6
Jul 16 23:07:05.314179 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:05.313797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" event={"ID":"542f46da-4e8a-48dc-acb4-b55405a05dd4","Type":"ContainerDied","Data":"77c2e021230f6a050a91df8df95c34eedbc84fbd0c51f689a34e2df5424978c1"}
Jul 16 23:07:05.314179 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:05.313901    2564 scope.go:117] "RemoveContainer" containerID="e87ccc7ce5ebad384320a1c3b4791f40b97f7ab77eab41ed4b43651929b6f6e0"
Jul 16 23:07:05.314179 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:05.314095    2564 scope.go:117] "RemoveContainer" containerID="77c2e021230f6a050a91df8df95c34eedbc84fbd0c51f689a34e2df5424978c1"
Jul 16 23:07:05.314386 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:05.314363    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq_odh-ai-gateway-infra(542f46da-4e8a-48dc-acb4-b55405a05dd4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4"
Jul 16 23:07:06.320634 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:06.320606    2564 scope.go:117] "RemoveContainer" containerID="77c2e021230f6a050a91df8df95c34eedbc84fbd0c51f689a34e2df5424978c1"
Jul 16 23:07:06.321040 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:06.320803    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq_odh-ai-gateway-infra(542f46da-4e8a-48dc-acb4-b55405a05dd4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4"
Jul 16 23:07:17.326285 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.326251    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"]
Jul 16 23:07:17.326672 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.326604    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:17.326672 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.326614    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f18b775-1292-4b7a-ada2-0a077e0f29dc" containerName="revoke-keys"
Jul 16 23:07:17.329435 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.329419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:17.340909 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.340877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"]
Jul 16 23:07:17.402024 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.401989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1acf98-fa46-4a7d-8d9a-ac75360cb697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds\" (UID: \"9e1acf98-fa46-4a7d-8d9a-ac75360cb697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:17.502587 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.502547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1acf98-fa46-4a7d-8d9a-ac75360cb697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds\" (UID: \"9e1acf98-fa46-4a7d-8d9a-ac75360cb697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:17.503240 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.503219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1acf98-fa46-4a7d-8d9a-ac75360cb697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds\" (UID: \"9e1acf98-fa46-4a7d-8d9a-ac75360cb697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:17.596387 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.596305    2564 scope.go:117] "RemoveContainer" containerID="77c2e021230f6a050a91df8df95c34eedbc84fbd0c51f689a34e2df5424978c1"
Jul 16 23:07:17.649110 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.649086    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:17.786594 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:17.786569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"]
Jul 16 23:07:17.788386 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:07:17.788356    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1acf98_fa46_4a7d_8d9a_ac75360cb697.slice/crio-82cb8b79f36506c6c32afda0a871d4c76e551d5fc184fc4572198426c66c0841 WatchSource:0}: Error finding container 82cb8b79f36506c6c32afda0a871d4c76e551d5fc184fc4572198426c66c0841: Status 404 returned error can't find the container with id 82cb8b79f36506c6c32afda0a871d4c76e551d5fc184fc4572198426c66c0841
Jul 16 23:07:18.386003 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.385970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" event={"ID":"9e1acf98-fa46-4a7d-8d9a-ac75360cb697","Type":"ContainerStarted","Data":"a4cc036311e0b2de56fcb7aa12c18d467b05b7d8867e15d4af416b91c9677502"}
Jul 16 23:07:18.386468 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.386030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" event={"ID":"9e1acf98-fa46-4a7d-8d9a-ac75360cb697","Type":"ContainerStarted","Data":"82cb8b79f36506c6c32afda0a871d4c76e551d5fc184fc4572198426c66c0841"}
Jul 16 23:07:18.387865 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.387835    2564 generic.go:356] "Generic (PLEG): container finished" podID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerID="c25930730b755eb7013433dd3482a1285390414b835945c4bf3f36cfa0b86d7a" exitCode=6
Jul 16 23:07:18.388002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.387902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" event={"ID":"542f46da-4e8a-48dc-acb4-b55405a05dd4","Type":"ContainerDied","Data":"c25930730b755eb7013433dd3482a1285390414b835945c4bf3f36cfa0b86d7a"}
Jul 16 23:07:18.388002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.387970    2564 scope.go:117] "RemoveContainer" containerID="77c2e021230f6a050a91df8df95c34eedbc84fbd0c51f689a34e2df5424978c1"
Jul 16 23:07:18.388257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.388242    2564 scope.go:117] "RemoveContainer" containerID="c25930730b755eb7013433dd3482a1285390414b835945c4bf3f36cfa0b86d7a"
Jul 16 23:07:18.388446 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:18.388426    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq_odh-ai-gateway-infra(542f46da-4e8a-48dc-acb4-b55405a05dd4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4"
Jul 16 23:07:18.405428 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:18.405388    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" podStartSLOduration=1.405375813 podStartE2EDuration="1.405375813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:07:18.404057149 +0000 UTC m=+2994.724613220" watchObservedRunningTime="2026-07-16 23:07:18.405375813 +0000 UTC m=+2994.725931884"
Jul 16 23:07:19.394832 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.394795    2564 generic.go:356] "Generic (PLEG): container finished" podID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerID="a4cc036311e0b2de56fcb7aa12c18d467b05b7d8867e15d4af416b91c9677502" exitCode=6
Jul 16 23:07:19.395395 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.394879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" event={"ID":"9e1acf98-fa46-4a7d-8d9a-ac75360cb697","Type":"ContainerDied","Data":"a4cc036311e0b2de56fcb7aa12c18d467b05b7d8867e15d4af416b91c9677502"}
Jul 16 23:07:19.395395 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.395099    2564 scope.go:117] "RemoveContainer" containerID="a4cc036311e0b2de56fcb7aa12c18d467b05b7d8867e15d4af416b91c9677502"
Jul 16 23:07:19.435979 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.435950    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"]
Jul 16 23:07:19.572409 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.572381    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:19.725652 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.725610    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/542f46da-4e8a-48dc-acb4-b55405a05dd4-maas-api-service-ca\") pod \"542f46da-4e8a-48dc-acb4-b55405a05dd4\" (UID: \"542f46da-4e8a-48dc-acb4-b55405a05dd4\") "
Jul 16 23:07:19.726004 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.725983    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/542f46da-4e8a-48dc-acb4-b55405a05dd4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "542f46da-4e8a-48dc-acb4-b55405a05dd4" (UID: "542f46da-4e8a-48dc-acb4-b55405a05dd4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:07:19.826333 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:19.826303    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/542f46da-4e8a-48dc-acb4-b55405a05dd4-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:07:20.404150 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.404042    2564 generic.go:356] "Generic (PLEG): container finished" podID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerID="16a358b2a47c2a72fbaa29a00de673965649b96c18b004badafd18a66f57bea7" exitCode=6
Jul 16 23:07:20.404615 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.404156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" event={"ID":"9e1acf98-fa46-4a7d-8d9a-ac75360cb697","Type":"ContainerDied","Data":"16a358b2a47c2a72fbaa29a00de673965649b96c18b004badafd18a66f57bea7"}
Jul 16 23:07:20.404615 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.404216    2564 scope.go:117] "RemoveContainer" containerID="a4cc036311e0b2de56fcb7aa12c18d467b05b7d8867e15d4af416b91c9677502"
Jul 16 23:07:20.404615 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.404377    2564 scope.go:117] "RemoveContainer" containerID="16a358b2a47c2a72fbaa29a00de673965649b96c18b004badafd18a66f57bea7"
Jul 16 23:07:20.404791 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:20.404616    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds_odh-ai-gateway-infra(9e1acf98-fa46-4a7d-8d9a-ac75360cb697)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697"
Jul 16 23:07:20.405634 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.405607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq" event={"ID":"542f46da-4e8a-48dc-acb4-b55405a05dd4","Type":"ContainerDied","Data":"5c19e893498eb56846e99b552bf9b987032a4c1cc29ec2bae59dac2e114ba9ad"}
Jul 16 23:07:20.405634 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.405636    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"
Jul 16 23:07:20.415303 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.415286    2564 scope.go:117] "RemoveContainer" containerID="c25930730b755eb7013433dd3482a1285390414b835945c4bf3f36cfa0b86d7a"
Jul 16 23:07:20.431943 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.431916    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"]
Jul 16 23:07:20.438277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:20.438248    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-jq4fq"]
Jul 16 23:07:21.412972 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:21.412943    2564 scope.go:117] "RemoveContainer" containerID="16a358b2a47c2a72fbaa29a00de673965649b96c18b004badafd18a66f57bea7"
Jul 16 23:07:21.413448 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:21.413227    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds_odh-ai-gateway-infra(9e1acf98-fa46-4a7d-8d9a-ac75360cb697)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697"
Jul 16 23:07:21.603504 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:21.603467    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" path="/var/lib/kubelet/pods/542f46da-4e8a-48dc-acb4-b55405a05dd4/volumes"
Jul 16 23:07:24.619172 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:24.619121    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 23:07:24.627779 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:24.627754    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 23:07:29.281899 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:29.281866    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:07:30.898116 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898079    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"]
Jul 16 23:07:30.898526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898437    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898449    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898459    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898465    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898474    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898526 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898480    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898736 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898547    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898736 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898556    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.898736 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.898562    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="542f46da-4e8a-48dc-acb4-b55405a05dd4" containerName="revoke-keys"
Jul 16 23:07:30.901344 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.901327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:30.909850 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:30.909823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"]
Jul 16 23:07:31.027950 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:31.027917    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67b12fde-6c0f-412a-b5b1-4edb0d4538bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp\" (UID: \"67b12fde-6c0f-412a-b5b1-4edb0d4538bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:31.129014 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:31.128972    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67b12fde-6c0f-412a-b5b1-4edb0d4538bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp\" (UID: \"67b12fde-6c0f-412a-b5b1-4edb0d4538bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:31.129626 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:31.129605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67b12fde-6c0f-412a-b5b1-4edb0d4538bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp\" (UID: \"67b12fde-6c0f-412a-b5b1-4edb0d4538bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:31.212778 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:31.212738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:31.347467 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:31.347438    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"]
Jul 16 23:07:31.349164 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:07:31.349116    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b12fde_6c0f_412a_b5b1_4edb0d4538bc.slice/crio-b4fb8925e8151cc9a08e7c3b09bc8f4bc7fd4bfe4a0b2ae006021fb4cfcfa008 WatchSource:0}: Error finding container b4fb8925e8151cc9a08e7c3b09bc8f4bc7fd4bfe4a0b2ae006021fb4cfcfa008: Status 404 returned error can't find the container with id b4fb8925e8151cc9a08e7c3b09bc8f4bc7fd4bfe4a0b2ae006021fb4cfcfa008
Jul 16 23:07:31.467801 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:31.467711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" event={"ID":"67b12fde-6c0f-412a-b5b1-4edb0d4538bc","Type":"ContainerStarted","Data":"b4fb8925e8151cc9a08e7c3b09bc8f4bc7fd4bfe4a0b2ae006021fb4cfcfa008"}
Jul 16 23:07:32.474989 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:32.474953    2564 generic.go:356] "Generic (PLEG): container finished" podID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerID="ca5ebe8fad36fa974fcf4d7cdf4136bb6f98ab5a4070d2fc3205816730648a16" exitCode=6
Jul 16 23:07:32.475400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:32.475016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" event={"ID":"67b12fde-6c0f-412a-b5b1-4edb0d4538bc","Type":"ContainerDied","Data":"ca5ebe8fad36fa974fcf4d7cdf4136bb6f98ab5a4070d2fc3205816730648a16"}
Jul 16 23:07:32.475400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:32.475283    2564 scope.go:117] "RemoveContainer" containerID="ca5ebe8fad36fa974fcf4d7cdf4136bb6f98ab5a4070d2fc3205816730648a16"
Jul 16 23:07:33.481519 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:33.481482    2564 generic.go:356] "Generic (PLEG): container finished" podID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerID="2d5f5c0db6730221cd0360af6db257cf4600da8fd90b3c3108a57d8c39ace055" exitCode=6
Jul 16 23:07:33.481951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:33.481575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" event={"ID":"67b12fde-6c0f-412a-b5b1-4edb0d4538bc","Type":"ContainerDied","Data":"2d5f5c0db6730221cd0360af6db257cf4600da8fd90b3c3108a57d8c39ace055"}
Jul 16 23:07:33.481951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:33.481633    2564 scope.go:117] "RemoveContainer" containerID="ca5ebe8fad36fa974fcf4d7cdf4136bb6f98ab5a4070d2fc3205816730648a16"
Jul 16 23:07:33.481951 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:33.481846    2564 scope.go:117] "RemoveContainer" containerID="2d5f5c0db6730221cd0360af6db257cf4600da8fd90b3c3108a57d8c39ace055"
Jul 16 23:07:33.482090 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:33.482069    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp_odh-ai-gateway-infra(67b12fde-6c0f-412a-b5b1-4edb0d4538bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc"
Jul 16 23:07:34.488576 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:34.488547    2564 scope.go:117] "RemoveContainer" containerID="2d5f5c0db6730221cd0360af6db257cf4600da8fd90b3c3108a57d8c39ace055"
Jul 16 23:07:34.488961 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:34.488791    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp_odh-ai-gateway-infra(67b12fde-6c0f-412a-b5b1-4edb0d4538bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc"
Jul 16 23:07:34.596711 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:34.596671    2564 scope.go:117] "RemoveContainer" containerID="16a358b2a47c2a72fbaa29a00de673965649b96c18b004badafd18a66f57bea7"
Jul 16 23:07:35.495234 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:35.495202    2564 generic.go:356] "Generic (PLEG): container finished" podID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerID="5e4c37e384c56f8135872807f7af47a90a45d04d355437406bac38491cf9a138" exitCode=6
Jul 16 23:07:35.495685 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:35.495276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" event={"ID":"9e1acf98-fa46-4a7d-8d9a-ac75360cb697","Type":"ContainerDied","Data":"5e4c37e384c56f8135872807f7af47a90a45d04d355437406bac38491cf9a138"}
Jul 16 23:07:35.495685 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:35.495337    2564 scope.go:117] "RemoveContainer" containerID="16a358b2a47c2a72fbaa29a00de673965649b96c18b004badafd18a66f57bea7"
Jul 16 23:07:35.495685 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:35.495629    2564 scope.go:117] "RemoveContainer" containerID="5e4c37e384c56f8135872807f7af47a90a45d04d355437406bac38491cf9a138"
Jul 16 23:07:35.495966 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:35.495832    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds_odh-ai-gateway-infra(9e1acf98-fa46-4a7d-8d9a-ac75360cb697)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697"
Jul 16 23:07:36.523057 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:36.523022    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"]
Jul 16 23:07:36.660599 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:36.660575    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:36.678301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:36.677917    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1acf98-fa46-4a7d-8d9a-ac75360cb697-maas-api-service-ca\") pod \"9e1acf98-fa46-4a7d-8d9a-ac75360cb697\" (UID: \"9e1acf98-fa46-4a7d-8d9a-ac75360cb697\") "
Jul 16 23:07:36.678301 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:36.678267    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1acf98-fa46-4a7d-8d9a-ac75360cb697-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e1acf98-fa46-4a7d-8d9a-ac75360cb697" (UID: "9e1acf98-fa46-4a7d-8d9a-ac75360cb697"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:07:36.678551 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:36.678364    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1acf98-fa46-4a7d-8d9a-ac75360cb697-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:07:37.509126 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:37.509097    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"
Jul 16 23:07:37.509327 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:37.509154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds" event={"ID":"9e1acf98-fa46-4a7d-8d9a-ac75360cb697","Type":"ContainerDied","Data":"82cb8b79f36506c6c32afda0a871d4c76e551d5fc184fc4572198426c66c0841"}
Jul 16 23:07:37.509327 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:37.509219    2564 scope.go:117] "RemoveContainer" containerID="5e4c37e384c56f8135872807f7af47a90a45d04d355437406bac38491cf9a138"
Jul 16 23:07:37.532406 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:37.532378    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"]
Jul 16 23:07:37.538964 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:37.538940    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-wk7ds"]
Jul 16 23:07:37.602893 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:37.602848    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" path="/var/lib/kubelet/pods/9e1acf98-fa46-4a7d-8d9a-ac75360cb697/volumes"
Jul 16 23:07:43.181982 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:43.181942    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:07:48.596313 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:48.596280    2564 scope.go:117] "RemoveContainer" containerID="2d5f5c0db6730221cd0360af6db257cf4600da8fd90b3c3108a57d8c39ace055"
Jul 16 23:07:49.576258 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:49.576160    2564 generic.go:356] "Generic (PLEG): container finished" podID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerID="d529cc269583a0bda8ea46f3bae1eb2f3f1845b36678746b1dc38430d070d544" exitCode=6
Jul 16 23:07:49.576418 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:49.576167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" event={"ID":"67b12fde-6c0f-412a-b5b1-4edb0d4538bc","Type":"ContainerDied","Data":"d529cc269583a0bda8ea46f3bae1eb2f3f1845b36678746b1dc38430d070d544"}
Jul 16 23:07:49.576418 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:49.576317    2564 scope.go:117] "RemoveContainer" containerID="2d5f5c0db6730221cd0360af6db257cf4600da8fd90b3c3108a57d8c39ace055"
Jul 16 23:07:49.576615 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:49.576596    2564 scope.go:117] "RemoveContainer" containerID="d529cc269583a0bda8ea46f3bae1eb2f3f1845b36678746b1dc38430d070d544"
Jul 16 23:07:49.576844 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:49.576825    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp_odh-ai-gateway-infra(67b12fde-6c0f-412a-b5b1-4edb0d4538bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc"
Jul 16 23:07:50.601740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:50.601706    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"]
Jul 16 23:07:50.737266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:50.737242    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:50.789967 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:50.789931    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67b12fde-6c0f-412a-b5b1-4edb0d4538bc-maas-api-service-ca\") pod \"67b12fde-6c0f-412a-b5b1-4edb0d4538bc\" (UID: \"67b12fde-6c0f-412a-b5b1-4edb0d4538bc\") "
Jul 16 23:07:50.790370 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:50.790351    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67b12fde-6c0f-412a-b5b1-4edb0d4538bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "67b12fde-6c0f-412a-b5b1-4edb0d4538bc" (UID: "67b12fde-6c0f-412a-b5b1-4edb0d4538bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:07:50.891415 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:50.891318    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67b12fde-6c0f-412a-b5b1-4edb0d4538bc-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:07:51.595910 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:51.595879    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"
Jul 16 23:07:51.602092 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:51.602068    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp" event={"ID":"67b12fde-6c0f-412a-b5b1-4edb0d4538bc","Type":"ContainerDied","Data":"b4fb8925e8151cc9a08e7c3b09bc8f4bc7fd4bfe4a0b2ae006021fb4cfcfa008"}
Jul 16 23:07:51.602503 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:51.602114    2564 scope.go:117] "RemoveContainer" containerID="d529cc269583a0bda8ea46f3bae1eb2f3f1845b36678746b1dc38430d070d544"
Jul 16 23:07:51.620840 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:51.620808    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"]
Jul 16 23:07:51.624495 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:51.624469    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-spsgp"]
Jul 16 23:07:52.690198 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690161    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"]
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690483    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690494    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690510    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690515    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690526    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690532    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690541    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690546    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690556    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:07:52.690582 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690561    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:07:52.690919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690609    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690618    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:07:52.690919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690627    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:07:52.690919 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.690632    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:07:52.695071 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.695046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:07:52.697326 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.697307    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:07:52.697421 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.697376    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:07:52.702262 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.702239    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"]
Jul 16 23:07:52.808679 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.808649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1388e51e-999b-46e9-bb54-bbc722780d14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7\" (UID: \"1388e51e-999b-46e9-bb54-bbc722780d14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:07:52.910037 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.910005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1388e51e-999b-46e9-bb54-bbc722780d14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7\" (UID: \"1388e51e-999b-46e9-bb54-bbc722780d14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:07:52.910661 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:52.910636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1388e51e-999b-46e9-bb54-bbc722780d14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7\" (UID: \"1388e51e-999b-46e9-bb54-bbc722780d14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:07:53.006275 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:53.006240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:07:53.137698 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:53.137670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"]
Jul 16 23:07:53.139289 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:07:53.139257    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1388e51e_999b_46e9_bb54_bbc722780d14.slice/crio-929a413b663e843f30d4f0c556a74fbd2f14c9e69accc2258a38eb39ec1f2854 WatchSource:0}: Error finding container 929a413b663e843f30d4f0c556a74fbd2f14c9e69accc2258a38eb39ec1f2854: Status 404 returned error can't find the container with id 929a413b663e843f30d4f0c556a74fbd2f14c9e69accc2258a38eb39ec1f2854
Jul 16 23:07:53.602073 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:53.602042    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" path="/var/lib/kubelet/pods/67b12fde-6c0f-412a-b5b1-4edb0d4538bc/volumes"
Jul 16 23:07:53.609060 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:53.609030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" event={"ID":"1388e51e-999b-46e9-bb54-bbc722780d14","Type":"ContainerStarted","Data":"cc04a412fea207dee3c9f2b5e966afdd499ce555781e9433039824cbcd8b38b5"}
Jul 16 23:07:53.609233 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:53.609092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" event={"ID":"1388e51e-999b-46e9-bb54-bbc722780d14","Type":"ContainerStarted","Data":"929a413b663e843f30d4f0c556a74fbd2f14c9e69accc2258a38eb39ec1f2854"}
Jul 16 23:07:53.624650 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:53.624599    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" podStartSLOduration=1.624584258 podStartE2EDuration="1.624584258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:07:53.622185317 +0000 UTC m=+3029.942741389" watchObservedRunningTime="2026-07-16 23:07:53.624584258 +0000 UTC m=+3029.945140332"
Jul 16 23:07:54.615864 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:54.615827    2564 generic.go:356] "Generic (PLEG): container finished" podID="1388e51e-999b-46e9-bb54-bbc722780d14" containerID="cc04a412fea207dee3c9f2b5e966afdd499ce555781e9433039824cbcd8b38b5" exitCode=6
Jul 16 23:07:54.616326 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:54.615921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" event={"ID":"1388e51e-999b-46e9-bb54-bbc722780d14","Type":"ContainerDied","Data":"cc04a412fea207dee3c9f2b5e966afdd499ce555781e9433039824cbcd8b38b5"}
Jul 16 23:07:54.616326 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:54.616068    2564 scope.go:117] "RemoveContainer" containerID="cc04a412fea207dee3c9f2b5e966afdd499ce555781e9433039824cbcd8b38b5"
Jul 16 23:07:55.623386 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:55.623287    2564 generic.go:356] "Generic (PLEG): container finished" podID="1388e51e-999b-46e9-bb54-bbc722780d14" containerID="6aea8fdcd7935ac9453cf1077b139e27dc119e526b0e65d60745ffdb752010d0" exitCode=6
Jul 16 23:07:55.623817 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:55.623376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" event={"ID":"1388e51e-999b-46e9-bb54-bbc722780d14","Type":"ContainerDied","Data":"6aea8fdcd7935ac9453cf1077b139e27dc119e526b0e65d60745ffdb752010d0"}
Jul 16 23:07:55.623817 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:55.623431    2564 scope.go:117] "RemoveContainer" containerID="cc04a412fea207dee3c9f2b5e966afdd499ce555781e9433039824cbcd8b38b5"
Jul 16 23:07:55.623817 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:55.623576    2564 scope.go:117] "RemoveContainer" containerID="6aea8fdcd7935ac9453cf1077b139e27dc119e526b0e65d60745ffdb752010d0"
Jul 16 23:07:55.623938 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:55.623827    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7_odh-ai-gateway-infra(1388e51e-999b-46e9-bb54-bbc722780d14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" podUID="1388e51e-999b-46e9-bb54-bbc722780d14"
Jul 16 23:07:56.631257 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:07:56.631229    2564 scope.go:117] "RemoveContainer" containerID="6aea8fdcd7935ac9453cf1077b139e27dc119e526b0e65d60745ffdb752010d0"
Jul 16 23:07:56.631637 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:07:56.631419    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7_odh-ai-gateway-infra(1388e51e-999b-46e9-bb54-bbc722780d14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" podUID="1388e51e-999b-46e9-bb54-bbc722780d14"
Jul 16 23:08:02.078885 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:02.078846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:08:10.597056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:10.597023    2564 scope.go:117] "RemoveContainer" containerID="6aea8fdcd7935ac9453cf1077b139e27dc119e526b0e65d60745ffdb752010d0"
Jul 16 23:08:11.713400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:11.713358    2564 generic.go:356] "Generic (PLEG): container finished" podID="1388e51e-999b-46e9-bb54-bbc722780d14" containerID="1262a91f36a08016db05a60dc1ba6c6e8dbe56520d8953dcbf25cf4f02c5c6d7" exitCode=6
Jul 16 23:08:11.713798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:11.713418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" event={"ID":"1388e51e-999b-46e9-bb54-bbc722780d14","Type":"ContainerDied","Data":"1262a91f36a08016db05a60dc1ba6c6e8dbe56520d8953dcbf25cf4f02c5c6d7"}
Jul 16 23:08:11.713798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:11.713469    2564 scope.go:117] "RemoveContainer" containerID="6aea8fdcd7935ac9453cf1077b139e27dc119e526b0e65d60745ffdb752010d0"
Jul 16 23:08:11.713798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:11.713758    2564 scope.go:117] "RemoveContainer" containerID="1262a91f36a08016db05a60dc1ba6c6e8dbe56520d8953dcbf25cf4f02c5c6d7"
Jul 16 23:08:11.714011 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:08:11.713990    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7_odh-ai-gateway-infra(1388e51e-999b-46e9-bb54-bbc722780d14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" podUID="1388e51e-999b-46e9-bb54-bbc722780d14"
Jul 16 23:08:12.739928 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:12.739892    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"]
Jul 16 23:08:12.869625 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:12.869589    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:08:13.002030 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.001936    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1388e51e-999b-46e9-bb54-bbc722780d14-maas-api-service-ca\") pod \"1388e51e-999b-46e9-bb54-bbc722780d14\" (UID: \"1388e51e-999b-46e9-bb54-bbc722780d14\") "
Jul 16 23:08:13.002354 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.002331    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1388e51e-999b-46e9-bb54-bbc722780d14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1388e51e-999b-46e9-bb54-bbc722780d14" (UID: "1388e51e-999b-46e9-bb54-bbc722780d14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:08:13.102660 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.102627    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1388e51e-999b-46e9-bb54-bbc722780d14-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:08:13.730471 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.730437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7" event={"ID":"1388e51e-999b-46e9-bb54-bbc722780d14","Type":"ContainerDied","Data":"929a413b663e843f30d4f0c556a74fbd2f14c9e69accc2258a38eb39ec1f2854"}
Jul 16 23:08:13.730665 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.730497    2564 scope.go:117] "RemoveContainer" containerID="1262a91f36a08016db05a60dc1ba6c6e8dbe56520d8953dcbf25cf4f02c5c6d7"
Jul 16 23:08:13.730665 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.730444    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"
Jul 16 23:08:13.746742 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.746706    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"]
Jul 16 23:08:13.752772 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:13.752743    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x7h7"]
Jul 16 23:08:15.601858 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:15.601778    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" path="/var/lib/kubelet/pods/1388e51e-999b-46e9-bb54-bbc722780d14/volumes"
Jul 16 23:08:17.375699 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.375661    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"]
Jul 16 23:08:17.376070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.375994    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.376070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376004    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.376070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376019    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.376070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376024    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.376070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376039    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:08:17.376070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376044    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:08:17.376322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376097    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="67b12fde-6c0f-412a-b5b1-4edb0d4538bc" containerName="revoke-keys"
Jul 16 23:08:17.376322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376105    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1acf98-fa46-4a7d-8d9a-ac75360cb697" containerName="revoke-keys"
Jul 16 23:08:17.376322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376111    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.376322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376117    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.376322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.376123    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:17.382165 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.382035    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:17.385670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.385646    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:08:17.386508 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.386485    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:08:17.388241 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.388216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"]
Jul 16 23:08:17.542801 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.542769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea162bd7-343e-4e96-bed5-436a16c4a662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm\" (UID: \"ea162bd7-343e-4e96-bed5-436a16c4a662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:17.643591 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.643502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea162bd7-343e-4e96-bed5-436a16c4a662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm\" (UID: \"ea162bd7-343e-4e96-bed5-436a16c4a662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:17.644067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.644047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea162bd7-343e-4e96-bed5-436a16c4a662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm\" (UID: \"ea162bd7-343e-4e96-bed5-436a16c4a662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:17.695155 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.695109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:17.826099 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:17.826074    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"]
Jul 16 23:08:17.828269 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:08:17.828238    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea162bd7_343e_4e96_bed5_436a16c4a662.slice/crio-e42979f3f67832e82fc1d0f26a66a95de0e6416dfb2efeebbc10af5b071ff9ce WatchSource:0}: Error finding container e42979f3f67832e82fc1d0f26a66a95de0e6416dfb2efeebbc10af5b071ff9ce: Status 404 returned error can't find the container with id e42979f3f67832e82fc1d0f26a66a95de0e6416dfb2efeebbc10af5b071ff9ce
Jul 16 23:08:18.761224 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:18.761119    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerID="0183f2c119778927762f06af50f69ff5db93fbe67e53f579b6066df566aeb9f8" exitCode=6
Jul 16 23:08:18.761224 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:18.761182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" event={"ID":"ea162bd7-343e-4e96-bed5-436a16c4a662","Type":"ContainerDied","Data":"0183f2c119778927762f06af50f69ff5db93fbe67e53f579b6066df566aeb9f8"}
Jul 16 23:08:18.761756 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:18.761234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" event={"ID":"ea162bd7-343e-4e96-bed5-436a16c4a662","Type":"ContainerStarted","Data":"e42979f3f67832e82fc1d0f26a66a95de0e6416dfb2efeebbc10af5b071ff9ce"}
Jul 16 23:08:18.761756 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:18.761453    2564 scope.go:117] "RemoveContainer" containerID="0183f2c119778927762f06af50f69ff5db93fbe67e53f579b6066df566aeb9f8"
Jul 16 23:08:19.072865 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:19.072831    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:08:19.767854 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:19.767750    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerID="c2d222518d5ea1e8956b8dc58af9d9c89dca1d12970fd7ffb684c1689193a737" exitCode=6
Jul 16 23:08:19.767854 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:19.767818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" event={"ID":"ea162bd7-343e-4e96-bed5-436a16c4a662","Type":"ContainerDied","Data":"c2d222518d5ea1e8956b8dc58af9d9c89dca1d12970fd7ffb684c1689193a737"}
Jul 16 23:08:19.768400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:19.767877    2564 scope.go:117] "RemoveContainer" containerID="0183f2c119778927762f06af50f69ff5db93fbe67e53f579b6066df566aeb9f8"
Jul 16 23:08:19.768400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:19.768085    2564 scope.go:117] "RemoveContainer" containerID="c2d222518d5ea1e8956b8dc58af9d9c89dca1d12970fd7ffb684c1689193a737"
Jul 16 23:08:19.768400 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:08:19.768352    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm_odh-ai-gateway-infra(ea162bd7-343e-4e96-bed5-436a16c4a662)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662"
Jul 16 23:08:20.775571 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:20.775546    2564 scope.go:117] "RemoveContainer" containerID="c2d222518d5ea1e8956b8dc58af9d9c89dca1d12970fd7ffb684c1689193a737"
Jul 16 23:08:20.776017 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:08:20.775793    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm_odh-ai-gateway-infra(ea162bd7-343e-4e96-bed5-436a16c4a662)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662"
Jul 16 23:08:32.596521 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:32.596487    2564 scope.go:117] "RemoveContainer" containerID="c2d222518d5ea1e8956b8dc58af9d9c89dca1d12970fd7ffb684c1689193a737"
Jul 16 23:08:33.859322 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:33.859286    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerID="4203b9a51acea80a04ec102a587b9b3a987512f215f18688b9617f0266466c21" exitCode=6
Jul 16 23:08:33.859732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:33.859359    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" event={"ID":"ea162bd7-343e-4e96-bed5-436a16c4a662","Type":"ContainerDied","Data":"4203b9a51acea80a04ec102a587b9b3a987512f215f18688b9617f0266466c21"}
Jul 16 23:08:33.859732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:33.859421    2564 scope.go:117] "RemoveContainer" containerID="c2d222518d5ea1e8956b8dc58af9d9c89dca1d12970fd7ffb684c1689193a737"
Jul 16 23:08:33.859732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:33.859717    2564 scope.go:117] "RemoveContainer" containerID="4203b9a51acea80a04ec102a587b9b3a987512f215f18688b9617f0266466c21"
Jul 16 23:08:33.859922 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:08:33.859908    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm_odh-ai-gateway-infra(ea162bd7-343e-4e96-bed5-436a16c4a662)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662"
Jul 16 23:08:34.885156 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:34.885110    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"]
Jul 16 23:08:35.015624 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.015594    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:35.105103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.105063    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea162bd7-343e-4e96-bed5-436a16c4a662-maas-api-service-ca\") pod \"ea162bd7-343e-4e96-bed5-436a16c4a662\" (UID: \"ea162bd7-343e-4e96-bed5-436a16c4a662\") "
Jul 16 23:08:35.105461 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.105439    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea162bd7-343e-4e96-bed5-436a16c4a662-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea162bd7-343e-4e96-bed5-436a16c4a662" (UID: "ea162bd7-343e-4e96-bed5-436a16c4a662"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:08:35.206355 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.206320    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea162bd7-343e-4e96-bed5-436a16c4a662-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:08:35.874053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.873975    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"
Jul 16 23:08:35.874224 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.873977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm" event={"ID":"ea162bd7-343e-4e96-bed5-436a16c4a662","Type":"ContainerDied","Data":"e42979f3f67832e82fc1d0f26a66a95de0e6416dfb2efeebbc10af5b071ff9ce"}
Jul 16 23:08:35.874224 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.874115    2564 scope.go:117] "RemoveContainer" containerID="4203b9a51acea80a04ec102a587b9b3a987512f215f18688b9617f0266466c21"
Jul 16 23:08:35.890585 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.890562    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"]
Jul 16 23:08:35.898670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:35.898641    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-n99bm"]
Jul 16 23:08:37.382315 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:37.382278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:08:37.602152 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:37.602111    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" path="/var/lib/kubelet/pods/ea162bd7-343e-4e96-bed5-436a16c4a662/volumes"
Jul 16 23:08:52.732385 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732351    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"]
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732691    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732701    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1388e51e-999b-46e9-bb54-bbc722780d14" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732709    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732715    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732734    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732740    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732786    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732795    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.732806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.732802    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:08:52.737200 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.737183    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:08:52.739700 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.739677    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:08:52.739700 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.739690    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:08:52.744394 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.744368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"]
Jul 16 23:08:52.870646 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.870613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287\" (UID: \"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:08:52.971287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.971255    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287\" (UID: \"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:08:52.971895 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:52.971877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287\" (UID: \"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:08:53.051333 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:53.051240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:08:53.187099 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:53.187068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"]
Jul 16 23:08:53.188474 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:08:53.188448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce48dc8e_d3de_4b82_a9e6_5cf329be2ed8.slice/crio-c34a69f1921e35ead5e3e3b58c9c4a05a7c0f3025e2c0b3c201cf4ae9d612f4b WatchSource:0}: Error finding container c34a69f1921e35ead5e3e3b58c9c4a05a7c0f3025e2c0b3c201cf4ae9d612f4b: Status 404 returned error can't find the container with id c34a69f1921e35ead5e3e3b58c9c4a05a7c0f3025e2c0b3c201cf4ae9d612f4b
Jul 16 23:08:53.971305 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:53.971270    2564 generic.go:356] "Generic (PLEG): container finished" podID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerID="dee392ceb1bb953e93f78facc6cfa070a0cfe7efcd35b1ced79d74357fdcc072" exitCode=6
Jul 16 23:08:53.971677 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:53.971355    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" event={"ID":"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8","Type":"ContainerDied","Data":"dee392ceb1bb953e93f78facc6cfa070a0cfe7efcd35b1ced79d74357fdcc072"}
Jul 16 23:08:53.971677 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:53.971411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" event={"ID":"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8","Type":"ContainerStarted","Data":"c34a69f1921e35ead5e3e3b58c9c4a05a7c0f3025e2c0b3c201cf4ae9d612f4b"}
Jul 16 23:08:53.971677 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:53.971496    2564 scope.go:117] "RemoveContainer" containerID="dee392ceb1bb953e93f78facc6cfa070a0cfe7efcd35b1ced79d74357fdcc072"
Jul 16 23:08:54.976638 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:54.976599    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:08:54.979756 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:54.979726    2564 generic.go:356] "Generic (PLEG): container finished" podID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerID="d25c330153a691556873c64ca4816182f171561e2ac15e556e82b310671fbc3b" exitCode=6
Jul 16 23:08:54.979892 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:54.979817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" event={"ID":"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8","Type":"ContainerDied","Data":"d25c330153a691556873c64ca4816182f171561e2ac15e556e82b310671fbc3b"}
Jul 16 23:08:54.979892 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:54.979871    2564 scope.go:117] "RemoveContainer" containerID="dee392ceb1bb953e93f78facc6cfa070a0cfe7efcd35b1ced79d74357fdcc072"
Jul 16 23:08:54.980003 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:54.979989    2564 scope.go:117] "RemoveContainer" containerID="d25c330153a691556873c64ca4816182f171561e2ac15e556e82b310671fbc3b"
Jul 16 23:08:54.980223 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:08:54.980200    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287_odh-ai-gateway-infra(ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8"
Jul 16 23:08:55.987244 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:08:55.987218    2564 scope.go:117] "RemoveContainer" containerID="d25c330153a691556873c64ca4816182f171561e2ac15e556e82b310671fbc3b"
Jul 16 23:08:55.987634 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:08:55.987430    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287_odh-ai-gateway-infra(ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8"
Jul 16 23:09:09.309440 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.309401    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"]
Jul 16 23:09:09.309950 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.309767    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:09:09.309950 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.309778    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea162bd7-343e-4e96-bed5-436a16c4a662" containerName="revoke-keys"
Jul 16 23:09:09.312768 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.312751    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:09.320593 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.320564    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"]
Jul 16 23:09:09.321071 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.321048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c05f4d6-a23b-4c2c-b1b7-939a4379b688-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-xbh2n\" (UID: \"7c05f4d6-a23b-4c2c-b1b7-939a4379b688\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:09.421707 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.421668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c05f4d6-a23b-4c2c-b1b7-939a4379b688-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-xbh2n\" (UID: \"7c05f4d6-a23b-4c2c-b1b7-939a4379b688\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:09.422364 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.422338    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c05f4d6-a23b-4c2c-b1b7-939a4379b688-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-xbh2n\" (UID: \"7c05f4d6-a23b-4c2c-b1b7-939a4379b688\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:09.625239 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.625156    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:09.756674 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:09.756645    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"]
Jul 16 23:09:09.758862 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:09:09.758819    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c05f4d6_a23b_4c2c_b1b7_939a4379b688.slice/crio-26ef4e9c2123e5a1545ec89bf303bc9ab4061a61066b32d746f1a0caffbe381d WatchSource:0}: Error finding container 26ef4e9c2123e5a1545ec89bf303bc9ab4061a61066b32d746f1a0caffbe381d: Status 404 returned error can't find the container with id 26ef4e9c2123e5a1545ec89bf303bc9ab4061a61066b32d746f1a0caffbe381d
Jul 16 23:09:10.063025 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:10.062986    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" event={"ID":"7c05f4d6-a23b-4c2c-b1b7-939a4379b688","Type":"ContainerStarted","Data":"26ef4e9c2123e5a1545ec89bf303bc9ab4061a61066b32d746f1a0caffbe381d"}
Jul 16 23:09:10.596193 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:10.596164    2564 scope.go:117] "RemoveContainer" containerID="d25c330153a691556873c64ca4816182f171561e2ac15e556e82b310671fbc3b"
Jul 16 23:09:11.072713 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:11.072623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" event={"ID":"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8","Type":"ContainerStarted","Data":"8bb18bc751ce897273057171f86f8876971af29df9703e6f581493dd203008be"}
Jul 16 23:09:11.074125 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:11.074100    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerID="f3c9d1a12b84e9eff47b7b8eb979d1f2c69c642de4dba0415d07a05ade0b97ec" exitCode=6
Jul 16 23:09:11.074273 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:11.074152    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" event={"ID":"7c05f4d6-a23b-4c2c-b1b7-939a4379b688","Type":"ContainerDied","Data":"f3c9d1a12b84e9eff47b7b8eb979d1f2c69c642de4dba0415d07a05ade0b97ec"}
Jul 16 23:09:11.074328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:11.074315    2564 scope.go:117] "RemoveContainer" containerID="f3c9d1a12b84e9eff47b7b8eb979d1f2c69c642de4dba0415d07a05ade0b97ec"
Jul 16 23:09:11.092048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:11.091993    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" podStartSLOduration=19.091975084 podStartE2EDuration="19.091975084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:09:11.086962559 +0000 UTC m=+3107.407518632" watchObservedRunningTime="2026-07-16 23:09:11.091975084 +0000 UTC m=+3107.412531156"
Jul 16 23:09:11.782831 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:11.782791    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:09:12.081055 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.081023    2564 generic.go:356] "Generic (PLEG): container finished" podID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerID="8bb18bc751ce897273057171f86f8876971af29df9703e6f581493dd203008be" exitCode=6
Jul 16 23:09:12.081259 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.081097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" event={"ID":"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8","Type":"ContainerDied","Data":"8bb18bc751ce897273057171f86f8876971af29df9703e6f581493dd203008be"}
Jul 16 23:09:12.081259 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.081165    2564 scope.go:117] "RemoveContainer" containerID="d25c330153a691556873c64ca4816182f171561e2ac15e556e82b310671fbc3b"
Jul 16 23:09:12.081489 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.081464    2564 scope.go:117] "RemoveContainer" containerID="8bb18bc751ce897273057171f86f8876971af29df9703e6f581493dd203008be"
Jul 16 23:09:12.081704 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:09:12.081682    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287_odh-ai-gateway-infra(ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8"
Jul 16 23:09:12.083007 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.082989    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerID="24b0044fde04b3ce7f2f68c63287e83d0f30ac47e060bf8156db1b5a383c5243" exitCode=6
Jul 16 23:09:12.083102 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.083068    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" event={"ID":"7c05f4d6-a23b-4c2c-b1b7-939a4379b688","Type":"ContainerDied","Data":"24b0044fde04b3ce7f2f68c63287e83d0f30ac47e060bf8156db1b5a383c5243"}
Jul 16 23:09:12.083255 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.083242    2564 scope.go:117] "RemoveContainer" containerID="24b0044fde04b3ce7f2f68c63287e83d0f30ac47e060bf8156db1b5a383c5243"
Jul 16 23:09:12.083413 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:09:12.083397    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-xbh2n_odh-ai-gateway-infra(7c05f4d6-a23b-4c2c-b1b7-939a4379b688)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688"
Jul 16 23:09:12.092541 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.092525    2564 scope.go:117] "RemoveContainer" containerID="f3c9d1a12b84e9eff47b7b8eb979d1f2c69c642de4dba0415d07a05ade0b97ec"
Jul 16 23:09:12.098597 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:12.098568    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"]
Jul 16 23:09:13.091826 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:13.091804    2564 scope.go:117] "RemoveContainer" containerID="24b0044fde04b3ce7f2f68c63287e83d0f30ac47e060bf8156db1b5a383c5243"
Jul 16 23:09:13.092120 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:09:13.092003    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-xbh2n_odh-ai-gateway-infra(7c05f4d6-a23b-4c2c-b1b7-939a4379b688)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688"
Jul 16 23:09:13.228907 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:13.228879    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:09:13.257233 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:13.257196    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8-maas-api-service-ca\") pod \"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8\" (UID: \"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8\") "
Jul 16 23:09:13.257532 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:13.257512    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" (UID: "ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:09:13.257588 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:13.257575    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:09:14.097701 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:14.097669    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"
Jul 16 23:09:14.098192 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:14.097702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287" event={"ID":"ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8","Type":"ContainerDied","Data":"c34a69f1921e35ead5e3e3b58c9c4a05a7c0f3025e2c0b3c201cf4ae9d612f4b"}
Jul 16 23:09:14.098192 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:14.097775    2564 scope.go:117] "RemoveContainer" containerID="8bb18bc751ce897273057171f86f8876971af29df9703e6f581493dd203008be"
Jul 16 23:09:14.114043 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:14.114017    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"]
Jul 16 23:09:14.119740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:14.119715    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-6x287"]
Jul 16 23:09:15.601877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:15.601839    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" path="/var/lib/kubelet/pods/ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8/volumes"
Jul 16 23:09:28.596463 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:28.596433    2564 scope.go:117] "RemoveContainer" containerID="24b0044fde04b3ce7f2f68c63287e83d0f30ac47e060bf8156db1b5a383c5243"
Jul 16 23:09:29.191057 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:29.190974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" event={"ID":"7c05f4d6-a23b-4c2c-b1b7-939a4379b688","Type":"ContainerStarted","Data":"06300d5624805bf083cd57000b7e117fb9967006f426ca49fa2333eed711dc7b"}
Jul 16 23:09:29.209520 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:29.209470    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" podStartSLOduration=20.209454813 podStartE2EDuration="20.209454813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:09:29.206527589 +0000 UTC m=+3125.527083664" watchObservedRunningTime="2026-07-16 23:09:29.209454813 +0000 UTC m=+3125.530010954"
Jul 16 23:09:30.198247 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:30.198214    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerID="06300d5624805bf083cd57000b7e117fb9967006f426ca49fa2333eed711dc7b" exitCode=6
Jul 16 23:09:30.198670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:30.198289    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" event={"ID":"7c05f4d6-a23b-4c2c-b1b7-939a4379b688","Type":"ContainerDied","Data":"06300d5624805bf083cd57000b7e117fb9967006f426ca49fa2333eed711dc7b"}
Jul 16 23:09:30.198670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:30.198345    2564 scope.go:117] "RemoveContainer" containerID="24b0044fde04b3ce7f2f68c63287e83d0f30ac47e060bf8156db1b5a383c5243"
Jul 16 23:09:30.198670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:30.198623    2564 scope.go:117] "RemoveContainer" containerID="06300d5624805bf083cd57000b7e117fb9967006f426ca49fa2333eed711dc7b"
Jul 16 23:09:30.198893 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:09:30.198853    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-xbh2n_odh-ai-gateway-infra(7c05f4d6-a23b-4c2c-b1b7-939a4379b688)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688"
Jul 16 23:09:30.224098 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:30.224063    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"]
Jul 16 23:09:30.688231 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:30.688147    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:09:31.338793 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:31.338751    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:31.421616 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:31.421577    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c05f4d6-a23b-4c2c-b1b7-939a4379b688-maas-api-service-ca\") pod \"7c05f4d6-a23b-4c2c-b1b7-939a4379b688\" (UID: \"7c05f4d6-a23b-4c2c-b1b7-939a4379b688\") "
Jul 16 23:09:31.421963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:31.421936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c05f4d6-a23b-4c2c-b1b7-939a4379b688-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7c05f4d6-a23b-4c2c-b1b7-939a4379b688" (UID: "7c05f4d6-a23b-4c2c-b1b7-939a4379b688"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:09:31.523274 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:31.523182    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c05f4d6-a23b-4c2c-b1b7-939a4379b688-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:09:32.211117 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.211085    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"
Jul 16 23:09:32.211336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.211084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n" event={"ID":"7c05f4d6-a23b-4c2c-b1b7-939a4379b688","Type":"ContainerDied","Data":"26ef4e9c2123e5a1545ec89bf303bc9ab4061a61066b32d746f1a0caffbe381d"}
Jul 16 23:09:32.211336 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.211220    2564 scope.go:117] "RemoveContainer" containerID="06300d5624805bf083cd57000b7e117fb9967006f426ca49fa2333eed711dc7b"
Jul 16 23:09:32.227558 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.227531    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"]
Jul 16 23:09:32.235570 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.235547    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-xbh2n"]
Jul 16 23:09:32.871292 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871265    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"]
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871621    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871636    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871650    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871655    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871662    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871667    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871676    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871681    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871692    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871698    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871707    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871714    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871776    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871787    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871792    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871798    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871804    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" containerName="revoke-keys"
Jul 16 23:09:32.871971 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.871809    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce48dc8e-d3de-4b82-a9e6-5cf329be2ed8" containerName="revoke-keys"
Jul 16 23:09:32.876390 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.876367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:32.880471 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.880407    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b5738f\"/\"openshift-service-ca.crt\""
Jul 16 23:09:32.880633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.880487    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5738f\"/\"sub-test-model-b5738f-kserve-self-signed-certs\""
Jul 16 23:09:32.880633 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.880396    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5738f\"/\"default-dockercfg-kmsqm\""
Jul 16 23:09:32.880759 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.880420    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b5738f\"/\"kube-root-ca.crt\""
Jul 16 23:09:32.883903 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.883878    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"]
Jul 16 23:09:32.937147 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.937114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-model-cache\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:32.937302 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.937236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a139939-c879-403e-bf84-b7938c6fd2d8-tls-certs\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:32.937302 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.937259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vm47\" (UniqueName: \"kubernetes.io/projected/7a139939-c879-403e-bf84-b7938c6fd2d8-kube-api-access-6vm47\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:32.937411 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.937373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-tmp-dir\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:32.937411 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.937405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-dshm\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:32.937480 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:32.937422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-home\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.039737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-dshm\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.039790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-home\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.039854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-model-cache\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.039934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a139939-c879-403e-bf84-b7938c6fd2d8-tls-certs\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.039960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vm47\" (UniqueName: \"kubernetes.io/projected/7a139939-c879-403e-bf84-b7938c6fd2d8-kube-api-access-6vm47\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.040054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-tmp-dir\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.040350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-home\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.040361    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-tmp-dir\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.040732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.040591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-model-cache\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.044926 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.044868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a139939-c879-403e-bf84-b7938c6fd2d8-tls-certs\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.044926 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.044877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-dshm\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.048855 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.048811    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vm47\" (UniqueName: \"kubernetes.io/projected/7a139939-c879-403e-bf84-b7938c6fd2d8-kube-api-access-6vm47\") pod \"sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") " pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.187668 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.187577    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:33.320460 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.320425    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"]
Jul 16 23:09:33.321224 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:09:33.321194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a139939_c879_403e_bf84_b7938c6fd2d8.slice/crio-d65577a505b7c40c04c05b0649ce2a6b52422b8825de62ade628581a940b23ae WatchSource:0}: Error finding container d65577a505b7c40c04c05b0649ce2a6b52422b8825de62ade628581a940b23ae: Status 404 returned error can't find the container with id d65577a505b7c40c04c05b0649ce2a6b52422b8825de62ade628581a940b23ae
Jul 16 23:09:33.323157 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.323126    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 23:09:33.604330 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:33.604083    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c05f4d6-a23b-4c2c-b1b7-939a4379b688" path="/var/lib/kubelet/pods/7c05f4d6-a23b-4c2c-b1b7-939a4379b688/volumes"
Jul 16 23:09:34.226250 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:34.226201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd" event={"ID":"7a139939-c879-403e-bf84-b7938c6fd2d8","Type":"ContainerStarted","Data":"d65577a505b7c40c04c05b0649ce2a6b52422b8825de62ade628581a940b23ae"}
Jul 16 23:09:35.232801 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:35.232768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd" event={"ID":"7a139939-c879-403e-bf84-b7938c6fd2d8","Type":"ContainerStarted","Data":"76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db"}
Jul 16 23:09:35.250735 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:35.250681    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd" podStartSLOduration=2.134295498 podStartE2EDuration="3.250664358s" totalImagesPullingTime="1.11636886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:09:35.248502737 +0000 UTC m=+3131.569058810" watchObservedRunningTime="2026-07-16 23:09:35.250664358 +0000 UTC m=+3131.571220430"
Jul 16 23:09:43.188626 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:43.188579    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:43.189214 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:43.188643    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:43.201103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:43.201072    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:43.289082 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:43.289053    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:09:47.092277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:47.092182    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:09:58.181470 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:09:58.181433    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:10:09.357683 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.357646    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"]
Jul 16 23:10:09.361057 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.361037    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:09.363315 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.363292    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:10:09.363439 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.363378    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:10:09.368073 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.368052    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"]
Jul 16 23:10:09.495833 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.495794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091e8ed6-0c93-4db7-8e74-758a18369416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-fj45j\" (UID: \"091e8ed6-0c93-4db7-8e74-758a18369416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:09.599072 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.599042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091e8ed6-0c93-4db7-8e74-758a18369416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-fj45j\" (UID: \"091e8ed6-0c93-4db7-8e74-758a18369416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:09.599717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.599696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091e8ed6-0c93-4db7-8e74-758a18369416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-fj45j\" (UID: \"091e8ed6-0c93-4db7-8e74-758a18369416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:09.673014 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:09.672919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:10.009920 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:10.009893    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"]
Jul 16 23:10:10.012254 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:10:10.012229    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091e8ed6_0c93_4db7_8e74_758a18369416.slice/crio-0ceabca31a657eb373403289b7c9d6f080c967ef71473578334cec742da10107 WatchSource:0}: Error finding container 0ceabca31a657eb373403289b7c9d6f080c967ef71473578334cec742da10107: Status 404 returned error can't find the container with id 0ceabca31a657eb373403289b7c9d6f080c967ef71473578334cec742da10107
Jul 16 23:10:10.420435 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:10.420399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" event={"ID":"091e8ed6-0c93-4db7-8e74-758a18369416","Type":"ContainerStarted","Data":"71c5200bb33ea47651e59caab0c2460b80d734dee6183152cdc8c29f027d061f"}
Jul 16 23:10:10.420854 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:10.420457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" event={"ID":"091e8ed6-0c93-4db7-8e74-758a18369416","Type":"ContainerStarted","Data":"0ceabca31a657eb373403289b7c9d6f080c967ef71473578334cec742da10107"}
Jul 16 23:10:10.437483 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:10.437426    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" podStartSLOduration=1.437403731 podStartE2EDuration="1.437403731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:10:10.434775895 +0000 UTC m=+3166.755331971" watchObservedRunningTime="2026-07-16 23:10:10.437403731 +0000 UTC m=+3166.757959803"
Jul 16 23:10:11.427056 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:11.427017    2564 generic.go:356] "Generic (PLEG): container finished" podID="091e8ed6-0c93-4db7-8e74-758a18369416" containerID="71c5200bb33ea47651e59caab0c2460b80d734dee6183152cdc8c29f027d061f" exitCode=6
Jul 16 23:10:11.427482 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:11.427106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" event={"ID":"091e8ed6-0c93-4db7-8e74-758a18369416","Type":"ContainerDied","Data":"71c5200bb33ea47651e59caab0c2460b80d734dee6183152cdc8c29f027d061f"}
Jul 16 23:10:11.427482 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:11.427258    2564 scope.go:117] "RemoveContainer" containerID="71c5200bb33ea47651e59caab0c2460b80d734dee6183152cdc8c29f027d061f"
Jul 16 23:10:12.434601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:12.434515    2564 generic.go:356] "Generic (PLEG): container finished" podID="091e8ed6-0c93-4db7-8e74-758a18369416" containerID="23724d0dd2fc4b752d491fb589131ae8a66947416b57210f996690539738a26d" exitCode=6
Jul 16 23:10:12.434601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:12.434587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" event={"ID":"091e8ed6-0c93-4db7-8e74-758a18369416","Type":"ContainerDied","Data":"23724d0dd2fc4b752d491fb589131ae8a66947416b57210f996690539738a26d"}
Jul 16 23:10:12.435067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:12.434630    2564 scope.go:117] "RemoveContainer" containerID="71c5200bb33ea47651e59caab0c2460b80d734dee6183152cdc8c29f027d061f"
Jul 16 23:10:12.435067 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:12.434789    2564 scope.go:117] "RemoveContainer" containerID="23724d0dd2fc4b752d491fb589131ae8a66947416b57210f996690539738a26d"
Jul 16 23:10:12.435067 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:10:12.435029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-fj45j_odh-ai-gateway-infra(091e8ed6-0c93-4db7-8e74-758a18369416)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" podUID="091e8ed6-0c93-4db7-8e74-758a18369416"
Jul 16 23:10:13.196585 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:13.196549    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:10:13.442213 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:13.442185    2564 scope.go:117] "RemoveContainer" containerID="23724d0dd2fc4b752d491fb589131ae8a66947416b57210f996690539738a26d"
Jul 16 23:10:13.442596 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:10:13.442397    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-fj45j_odh-ai-gateway-infra(091e8ed6-0c93-4db7-8e74-758a18369416)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" podUID="091e8ed6-0c93-4db7-8e74-758a18369416"
Jul 16 23:10:27.596215 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:27.596177    2564 scope.go:117] "RemoveContainer" containerID="23724d0dd2fc4b752d491fb589131ae8a66947416b57210f996690539738a26d"
Jul 16 23:10:28.527318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:28.527230    2564 generic.go:356] "Generic (PLEG): container finished" podID="091e8ed6-0c93-4db7-8e74-758a18369416" containerID="a4c6b20650e916578eea69843e8f777eedb4c9c58a42509dd57cc7e20e2b1910" exitCode=6
Jul 16 23:10:28.527318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:28.527307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" event={"ID":"091e8ed6-0c93-4db7-8e74-758a18369416","Type":"ContainerDied","Data":"a4c6b20650e916578eea69843e8f777eedb4c9c58a42509dd57cc7e20e2b1910"}
Jul 16 23:10:28.527517 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:28.527378    2564 scope.go:117] "RemoveContainer" containerID="23724d0dd2fc4b752d491fb589131ae8a66947416b57210f996690539738a26d"
Jul 16 23:10:28.527681 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:28.527665    2564 scope.go:117] "RemoveContainer" containerID="a4c6b20650e916578eea69843e8f777eedb4c9c58a42509dd57cc7e20e2b1910"
Jul 16 23:10:28.527957 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:10:28.527930    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-fj45j_odh-ai-gateway-infra(091e8ed6-0c93-4db7-8e74-758a18369416)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" podUID="091e8ed6-0c93-4db7-8e74-758a18369416"
Jul 16 23:10:28.643790 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:28.643759    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"]
Jul 16 23:10:29.551261 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:29.551225    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"]
Jul 16 23:10:29.679559 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:29.679530    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:29.788178 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:29.788104    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091e8ed6-0c93-4db7-8e74-758a18369416-maas-api-service-ca\") pod \"091e8ed6-0c93-4db7-8e74-758a18369416\" (UID: \"091e8ed6-0c93-4db7-8e74-758a18369416\") "
Jul 16 23:10:29.788489 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:29.788456    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091e8ed6-0c93-4db7-8e74-758a18369416-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "091e8ed6-0c93-4db7-8e74-758a18369416" (UID: "091e8ed6-0c93-4db7-8e74-758a18369416"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:10:29.889316 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:29.889219    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091e8ed6-0c93-4db7-8e74-758a18369416-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:30.541549 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:30.541521    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"
Jul 16 23:10:30.541736 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:30.541563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j" event={"ID":"091e8ed6-0c93-4db7-8e74-758a18369416","Type":"ContainerDied","Data":"0ceabca31a657eb373403289b7c9d6f080c967ef71473578334cec742da10107"}
Jul 16 23:10:30.541736 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:30.541623    2564 scope.go:117] "RemoveContainer" containerID="a4c6b20650e916578eea69843e8f777eedb4c9c58a42509dd57cc7e20e2b1910"
Jul 16 23:10:30.567769 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:30.567732    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"]
Jul 16 23:10:30.573807 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:30.573777    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-fj45j"]
Jul 16 23:10:31.602334 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:31.602295    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" path="/var/lib/kubelet/pods/091e8ed6-0c93-4db7-8e74-758a18369416/volumes"
Jul 16 23:10:32.685714 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:32.685669    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:10:41.880120 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880089    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"]
Jul 16 23:10:41.880499 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880450    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.880499 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880462    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.880499 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880489    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.880499 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880495    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.880648 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880547    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.880648 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880554    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.880648 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.880562    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:10:41.882755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.882739    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:41.884807 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.884783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5738f\"/\"rate-test-model-b5738f-kserve-self-signed-certs\""
Jul 16 23:10:41.893565 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.893537    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"]
Jul 16 23:10:41.996248 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.996205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tls-certs\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:41.996432 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.996377    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-model-cache\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:41.996481 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.996430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tmp-dir\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:41.996517 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.996488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb5zw\" (UniqueName: \"kubernetes.io/projected/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-kube-api-access-lb5zw\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:41.996553 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.996517    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-home\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:41.996553 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:41.996547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-dshm\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097086 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-model-cache\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097086 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tmp-dir\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb5zw\" (UniqueName: \"kubernetes.io/projected/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-kube-api-access-lb5zw\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-home\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-dshm\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097472 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tls-certs\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097547 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097525    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-model-cache\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097573    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-home\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.097610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.097588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tmp-dir\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.099659 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.099631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-dshm\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.099854 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.099836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tls-certs\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.104929 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.104909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb5zw\" (UniqueName: \"kubernetes.io/projected/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-kube-api-access-lb5zw\") pod \"rate-test-model-b5738f-kserve-d74d868f5-7kpkh\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") " pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.192952 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.192861    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:42.325909 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.325877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"]
Jul 16 23:10:42.327202 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:10:42.327173    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19fdce9d_7230_4a5d_82eb_2eb3cbf08049.slice/crio-72f629e1610294bab98f4bf9ac5aa9407468c5d13bdd5e4bb970dc89b302090f WatchSource:0}: Error finding container 72f629e1610294bab98f4bf9ac5aa9407468c5d13bdd5e4bb970dc89b302090f: Status 404 returned error can't find the container with id 72f629e1610294bab98f4bf9ac5aa9407468c5d13bdd5e4bb970dc89b302090f
Jul 16 23:10:42.612596 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.612558    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh" event={"ID":"19fdce9d-7230-4a5d-82eb-2eb3cbf08049","Type":"ContainerStarted","Data":"682a0780288816ec9e5e9691b87befc19a65730518b3b9e23354043b4ae073cf"}
Jul 16 23:10:42.612596 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.612603    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh" event={"ID":"19fdce9d-7230-4a5d-82eb-2eb3cbf08049","Type":"ContainerStarted","Data":"72f629e1610294bab98f4bf9ac5aa9407468c5d13bdd5e4bb970dc89b302090f"}
Jul 16 23:10:42.630307 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:42.630251    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh" podStartSLOduration=1.63023434 podStartE2EDuration="1.63023434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:10:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:10:42.627973854 +0000 UTC m=+3198.948529926" watchObservedRunningTime="2026-07-16 23:10:42.63023434 +0000 UTC m=+3198.950790412"
Jul 16 23:10:43.664190 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:43.664114    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd" podUID="7a139939-c879-403e-bf84-b7938c6fd2d8" containerName="main" containerID="cri-o://76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db" gracePeriod=45
Jul 16 23:10:43.911399 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:43.911370    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:10:44.016345 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-model-cache\") pod \"7a139939-c879-403e-bf84-b7938c6fd2d8\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") "
Jul 16 23:10:44.016521 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016376    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-dshm\") pod \"7a139939-c879-403e-bf84-b7938c6fd2d8\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") "
Jul 16 23:10:44.016521 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016433    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vm47\" (UniqueName: \"kubernetes.io/projected/7a139939-c879-403e-bf84-b7938c6fd2d8-kube-api-access-6vm47\") pod \"7a139939-c879-403e-bf84-b7938c6fd2d8\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") "
Jul 16 23:10:44.016521 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016483    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-home\") pod \"7a139939-c879-403e-bf84-b7938c6fd2d8\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") "
Jul 16 23:10:44.016521 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016506    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-tmp-dir\") pod \"7a139939-c879-403e-bf84-b7938c6fd2d8\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") "
Jul 16 23:10:44.016521 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016523    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a139939-c879-403e-bf84-b7938c6fd2d8-tls-certs\") pod \"7a139939-c879-403e-bf84-b7938c6fd2d8\" (UID: \"7a139939-c879-403e-bf84-b7938c6fd2d8\") "
Jul 16 23:10:44.016734 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016635    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a139939-c879-403e-bf84-b7938c6fd2d8" (UID: "7a139939-c879-403e-bf84-b7938c6fd2d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:10:44.016796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016765    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a139939-c879-403e-bf84-b7938c6fd2d8" (UID: "7a139939-c879-403e-bf84-b7938c6fd2d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:10:44.016796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016782    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-home" (OuterVolumeSpecName: "home") pod "7a139939-c879-403e-bf84-b7938c6fd2d8" (UID: "7a139939-c879-403e-bf84-b7938c6fd2d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:10:44.016912 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016882    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-home\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:44.016912 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016895    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-tmp-dir\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:44.016912 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.016904    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-model-cache\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:44.018667 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.018636    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a139939-c879-403e-bf84-b7938c6fd2d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a139939-c879-403e-bf84-b7938c6fd2d8" (UID: "7a139939-c879-403e-bf84-b7938c6fd2d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:10:44.018798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.018689    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a139939-c879-403e-bf84-b7938c6fd2d8-kube-api-access-6vm47" (OuterVolumeSpecName: "kube-api-access-6vm47") pod "7a139939-c879-403e-bf84-b7938c6fd2d8" (UID: "7a139939-c879-403e-bf84-b7938c6fd2d8"). InnerVolumeSpecName "kube-api-access-6vm47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:10:44.018798 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.018722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-dshm" (OuterVolumeSpecName: "dshm") pod "7a139939-c879-403e-bf84-b7938c6fd2d8" (UID: "7a139939-c879-403e-bf84-b7938c6fd2d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:10:44.118338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.118301    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vm47\" (UniqueName: \"kubernetes.io/projected/7a139939-c879-403e-bf84-b7938c6fd2d8-kube-api-access-6vm47\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:44.118338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.118333    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a139939-c879-403e-bf84-b7938c6fd2d8-tls-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:44.118338 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.118342    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a139939-c879-403e-bf84-b7938c6fd2d8-dshm\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:10:44.624995 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.624959    2564 generic.go:356] "Generic (PLEG): container finished" podID="7a139939-c879-403e-bf84-b7938c6fd2d8" containerID="76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db" exitCode=0
Jul 16 23:10:44.624995 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.625000    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"
Jul 16 23:10:44.625266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.625053    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd" event={"ID":"7a139939-c879-403e-bf84-b7938c6fd2d8","Type":"ContainerDied","Data":"76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db"}
Jul 16 23:10:44.625266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.625089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd" event={"ID":"7a139939-c879-403e-bf84-b7938c6fd2d8","Type":"ContainerDied","Data":"d65577a505b7c40c04c05b0649ce2a6b52422b8825de62ade628581a940b23ae"}
Jul 16 23:10:44.625266 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.625111    2564 scope.go:117] "RemoveContainer" containerID="76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db"
Jul 16 23:10:44.634693 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.634673    2564 scope.go:117] "RemoveContainer" containerID="76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db"
Jul 16 23:10:44.634971 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:10:44.634953    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db\": container with ID starting with 76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db not found: ID does not exist" containerID="76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db"
Jul 16 23:10:44.635016 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.634979    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db"} err="failed to get container status \"76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db\": rpc error: code = NotFound desc = could not find container \"76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db\": container with ID starting with 76b7d57d1873dbd9d945c8186c1786d8339cad20a9e89cf4cf79dc8139e055db not found: ID does not exist"
Jul 16 23:10:44.646648 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.646624    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"]
Jul 16 23:10:44.650053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:44.650029    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/sub-test-model-b5738f-kserve-5bb66d7f7c-2srtd"]
Jul 16 23:10:45.602449 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:45.602417    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a139939-c879-403e-bf84-b7938c6fd2d8" path="/var/lib/kubelet/pods/7a139939-c879-403e-bf84-b7938c6fd2d8/volumes"
Jul 16 23:10:48.081207 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:48.081172    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:10:52.193235 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:52.193193    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:52.193690 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:52.193255    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:52.205849 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:52.205820    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:10:52.679952 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:10:52.679918    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:11:23.686042 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:23.685953    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:11:44.881101 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:44.881061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:11:47.535822 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.535782    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"]
Jul 16 23:11:47.536552 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.536529    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a139939-c879-403e-bf84-b7938c6fd2d8" containerName="main"
Jul 16 23:11:47.536640 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.536557    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a139939-c879-403e-bf84-b7938c6fd2d8" containerName="main"
Jul 16 23:11:47.536640 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.536593    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:11:47.536640 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.536602    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="091e8ed6-0c93-4db7-8e74-758a18369416" containerName="revoke-keys"
Jul 16 23:11:47.536784 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.536696    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a139939-c879-403e-bf84-b7938c6fd2d8" containerName="main"
Jul 16 23:11:47.540159 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.540114    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:11:47.543222 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.543050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:11:47.543222 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.543049    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:11:47.545668 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.545640    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"]
Jul 16 23:11:47.709751 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.709708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa44d3-622a-49b3-b39b-dbaee30813aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr\" (UID: \"55fa44d3-622a-49b3-b39b-dbaee30813aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:11:47.810877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.810786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa44d3-622a-49b3-b39b-dbaee30813aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr\" (UID: \"55fa44d3-622a-49b3-b39b-dbaee30813aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:11:47.811427 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.811410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa44d3-622a-49b3-b39b-dbaee30813aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr\" (UID: \"55fa44d3-622a-49b3-b39b-dbaee30813aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:11:47.856931 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.856900    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:11:47.993065 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:47.992927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"]
Jul 16 23:11:47.995813 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:11:47.995785    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fa44d3_622a_49b3_b39b_dbaee30813aa.slice/crio-cf3274913e0e98d961698e8d098c1adba4b4f02d272bcf6b5e7a44b0850662c7 WatchSource:0}: Error finding container cf3274913e0e98d961698e8d098c1adba4b4f02d272bcf6b5e7a44b0850662c7: Status 404 returned error can't find the container with id cf3274913e0e98d961698e8d098c1adba4b4f02d272bcf6b5e7a44b0850662c7
Jul 16 23:11:48.968420 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:48.968386    2564 generic.go:356] "Generic (PLEG): container finished" podID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerID="86a1349fff98877793b4a3aebc5168b4fbf6355c9cd48851fe42356bb4aa2232" exitCode=6
Jul 16 23:11:48.968819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:48.968477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" event={"ID":"55fa44d3-622a-49b3-b39b-dbaee30813aa","Type":"ContainerDied","Data":"86a1349fff98877793b4a3aebc5168b4fbf6355c9cd48851fe42356bb4aa2232"}
Jul 16 23:11:48.968819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:48.968534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" event={"ID":"55fa44d3-622a-49b3-b39b-dbaee30813aa","Type":"ContainerStarted","Data":"cf3274913e0e98d961698e8d098c1adba4b4f02d272bcf6b5e7a44b0850662c7"}
Jul 16 23:11:48.968819 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:48.968653    2564 scope.go:117] "RemoveContainer" containerID="86a1349fff98877793b4a3aebc5168b4fbf6355c9cd48851fe42356bb4aa2232"
Jul 16 23:11:49.976298 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:49.976264    2564 generic.go:356] "Generic (PLEG): container finished" podID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerID="ba57261e4f901b0545a1636a9cb1216dc23fbbe6c117c7fd5f4cdd443debca21" exitCode=6
Jul 16 23:11:49.976806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:49.976346    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" event={"ID":"55fa44d3-622a-49b3-b39b-dbaee30813aa","Type":"ContainerDied","Data":"ba57261e4f901b0545a1636a9cb1216dc23fbbe6c117c7fd5f4cdd443debca21"}
Jul 16 23:11:49.976806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:49.976408    2564 scope.go:117] "RemoveContainer" containerID="86a1349fff98877793b4a3aebc5168b4fbf6355c9cd48851fe42356bb4aa2232"
Jul 16 23:11:49.976806 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:49.976650    2564 scope.go:117] "RemoveContainer" containerID="ba57261e4f901b0545a1636a9cb1216dc23fbbe6c117c7fd5f4cdd443debca21"
Jul 16 23:11:49.977088 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:11:49.976894    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr_odh-ai-gateway-infra(55fa44d3-622a-49b3-b39b-dbaee30813aa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa"
Jul 16 23:11:50.984363 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:11:50.984334    2564 scope.go:117] "RemoveContainer" containerID="ba57261e4f901b0545a1636a9cb1216dc23fbbe6c117c7fd5f4cdd443debca21"
Jul 16 23:11:50.984770 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:11:50.984536    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr_odh-ai-gateway-infra(55fa44d3-622a-49b3-b39b-dbaee30813aa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa"
Jul 16 23:12:00.283992 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:00.283951    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:12:04.596158 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:04.596098    2564 scope.go:117] "RemoveContainer" containerID="ba57261e4f901b0545a1636a9cb1216dc23fbbe6c117c7fd5f4cdd443debca21"
Jul 16 23:12:05.063958 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:05.063918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" event={"ID":"55fa44d3-622a-49b3-b39b-dbaee30813aa","Type":"ContainerStarted","Data":"ab8121bb47f72c8dc32965208226a94f19e6fa533c023f665819e27f769e59cd"}
Jul 16 23:12:05.079081 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:05.079028    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" podStartSLOduration=18.079012251 podStartE2EDuration="18.079012251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:12:05.077954026 +0000 UTC m=+3281.398510097" watchObservedRunningTime="2026-07-16 23:12:05.079012251 +0000 UTC m=+3281.399568322"
Jul 16 23:12:06.071739 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:06.071705    2564 generic.go:356] "Generic (PLEG): container finished" podID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerID="ab8121bb47f72c8dc32965208226a94f19e6fa533c023f665819e27f769e59cd" exitCode=6
Jul 16 23:12:06.072227 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:06.071777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" event={"ID":"55fa44d3-622a-49b3-b39b-dbaee30813aa","Type":"ContainerDied","Data":"ab8121bb47f72c8dc32965208226a94f19e6fa533c023f665819e27f769e59cd"}
Jul 16 23:12:06.072227 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:06.071841    2564 scope.go:117] "RemoveContainer" containerID="ba57261e4f901b0545a1636a9cb1216dc23fbbe6c117c7fd5f4cdd443debca21"
Jul 16 23:12:06.072227 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:06.072191    2564 scope.go:117] "RemoveContainer" containerID="ab8121bb47f72c8dc32965208226a94f19e6fa533c023f665819e27f769e59cd"
Jul 16 23:12:06.072440 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:12:06.072416    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr_odh-ai-gateway-infra(55fa44d3-622a-49b3-b39b-dbaee30813aa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa"
Jul 16 23:12:06.089617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:06.089561    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"]
Jul 16 23:12:07.217780 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:07.217756    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:12:07.285603 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:07.285571    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa44d3-622a-49b3-b39b-dbaee30813aa-maas-api-service-ca\") pod \"55fa44d3-622a-49b3-b39b-dbaee30813aa\" (UID: \"55fa44d3-622a-49b3-b39b-dbaee30813aa\") "
Jul 16 23:12:07.285940 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:07.285917    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55fa44d3-622a-49b3-b39b-dbaee30813aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55fa44d3-622a-49b3-b39b-dbaee30813aa" (UID: "55fa44d3-622a-49b3-b39b-dbaee30813aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:12:07.386742 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:07.386640    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa44d3-622a-49b3-b39b-dbaee30813aa-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:07.955705 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:07.955669    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"]
Jul 16 23:12:08.088683 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:08.088652    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"
Jul 16 23:12:08.088683 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:08.088670    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr" event={"ID":"55fa44d3-622a-49b3-b39b-dbaee30813aa","Type":"ContainerDied","Data":"cf3274913e0e98d961698e8d098c1adba4b4f02d272bcf6b5e7a44b0850662c7"}
Jul 16 23:12:08.088940 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:08.088726    2564 scope.go:117] "RemoveContainer" containerID="ab8121bb47f72c8dc32965208226a94f19e6fa533c023f665819e27f769e59cd"
Jul 16 23:12:08.106778 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:08.106737    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"]
Jul 16 23:12:08.111950 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:08.111926    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-r9zrr"]
Jul 16 23:12:09.453717 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.453683    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"]
Jul 16 23:12:09.454103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.454021    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:09.454103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.454032    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:09.454103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.454050    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:09.454103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.454056    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:09.454287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.454113    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:09.454287 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.454122    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:09.457765 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.457748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:09.460368 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.460320    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:12:09.460504 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.460470    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:12:09.464477 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.464454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"]
Jul 16 23:12:09.507487 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.507450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11666424-0504-402f-ac84-8c2d6cce9712-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-7s6vs\" (UID: \"11666424-0504-402f-ac84-8c2d6cce9712\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:09.602551 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.602521    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" path="/var/lib/kubelet/pods/55fa44d3-622a-49b3-b39b-dbaee30813aa/volumes"
Jul 16 23:12:09.608452 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.608431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11666424-0504-402f-ac84-8c2d6cce9712-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-7s6vs\" (UID: \"11666424-0504-402f-ac84-8c2d6cce9712\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:09.609011 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.608991    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11666424-0504-402f-ac84-8c2d6cce9712-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-7s6vs\" (UID: \"11666424-0504-402f-ac84-8c2d6cce9712\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:09.769558 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.769468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:09.902689 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:09.902664    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"]
Jul 16 23:12:09.904338 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:12:09.904314    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11666424_0504_402f_ac84_8c2d6cce9712.slice/crio-b5d709aa5aa6cf0ffce757294819408a91834648a37d3ab47cd4e22aae4ee0c5 WatchSource:0}: Error finding container b5d709aa5aa6cf0ffce757294819408a91834648a37d3ab47cd4e22aae4ee0c5: Status 404 returned error can't find the container with id b5d709aa5aa6cf0ffce757294819408a91834648a37d3ab47cd4e22aae4ee0c5
Jul 16 23:12:10.102921 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:10.102891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" event={"ID":"11666424-0504-402f-ac84-8c2d6cce9712","Type":"ContainerStarted","Data":"b5d709aa5aa6cf0ffce757294819408a91834648a37d3ab47cd4e22aae4ee0c5"}
Jul 16 23:12:11.109832 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:11.109799    2564 generic.go:356] "Generic (PLEG): container finished" podID="11666424-0504-402f-ac84-8c2d6cce9712" containerID="220e12e5e682f7512ae2fb156ccd16f5e2c224f06d374542883b7eaec30d0d6b" exitCode=6
Jul 16 23:12:11.110262 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:11.109892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" event={"ID":"11666424-0504-402f-ac84-8c2d6cce9712","Type":"ContainerDied","Data":"220e12e5e682f7512ae2fb156ccd16f5e2c224f06d374542883b7eaec30d0d6b"}
Jul 16 23:12:11.110262 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:11.110073    2564 scope.go:117] "RemoveContainer" containerID="220e12e5e682f7512ae2fb156ccd16f5e2c224f06d374542883b7eaec30d0d6b"
Jul 16 23:12:12.124298 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:12.124266    2564 generic.go:356] "Generic (PLEG): container finished" podID="11666424-0504-402f-ac84-8c2d6cce9712" containerID="ef84a3d47e1785ece0bbf8eebce39f333a545bc6f7a6640093f7b3c3c79a2f66" exitCode=6
Jul 16 23:12:12.124755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:12.124310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" event={"ID":"11666424-0504-402f-ac84-8c2d6cce9712","Type":"ContainerDied","Data":"ef84a3d47e1785ece0bbf8eebce39f333a545bc6f7a6640093f7b3c3c79a2f66"}
Jul 16 23:12:12.124755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:12.124389    2564 scope.go:117] "RemoveContainer" containerID="220e12e5e682f7512ae2fb156ccd16f5e2c224f06d374542883b7eaec30d0d6b"
Jul 16 23:12:12.124755 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:12.124657    2564 scope.go:117] "RemoveContainer" containerID="ef84a3d47e1785ece0bbf8eebce39f333a545bc6f7a6640093f7b3c3c79a2f66"
Jul 16 23:12:12.124920 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:12:12.124898    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-7s6vs_odh-ai-gateway-infra(11666424-0504-402f-ac84-8c2d6cce9712)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" podUID="11666424-0504-402f-ac84-8c2d6cce9712"
Jul 16 23:12:13.131305 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:13.131278    2564 scope.go:117] "RemoveContainer" containerID="ef84a3d47e1785ece0bbf8eebce39f333a545bc6f7a6640093f7b3c3c79a2f66"
Jul 16 23:12:13.131685 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:12:13.131502    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-7s6vs_odh-ai-gateway-infra(11666424-0504-402f-ac84-8c2d6cce9712)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" podUID="11666424-0504-402f-ac84-8c2d6cce9712"
Jul 16 23:12:14.781016 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:14.780976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:12:22.976064 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:22.976013    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh" podUID="19fdce9d-7230-4a5d-82eb-2eb3cbf08049" containerName="main" containerID="cri-o://682a0780288816ec9e5e9691b87befc19a65730518b3b9e23354043b4ae073cf" gracePeriod=45
Jul 16 23:12:23.187848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.187815    2564 generic.go:356] "Generic (PLEG): container finished" podID="19fdce9d-7230-4a5d-82eb-2eb3cbf08049" containerID="682a0780288816ec9e5e9691b87befc19a65730518b3b9e23354043b4ae073cf" exitCode=0
Jul 16 23:12:23.188030 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.187887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh" event={"ID":"19fdce9d-7230-4a5d-82eb-2eb3cbf08049","Type":"ContainerDied","Data":"682a0780288816ec9e5e9691b87befc19a65730518b3b9e23354043b4ae073cf"}
Jul 16 23:12:23.237460 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.237394    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:12:23.335572 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335539    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tls-certs\") pod \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") "
Jul 16 23:12:23.335733 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335592    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-home\") pod \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") "
Jul 16 23:12:23.335733 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335632    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb5zw\" (UniqueName: \"kubernetes.io/projected/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-kube-api-access-lb5zw\") pod \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") "
Jul 16 23:12:23.335733 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335650    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-dshm\") pod \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") "
Jul 16 23:12:23.335733 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335716    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-model-cache\") pod \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") "
Jul 16 23:12:23.336050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335741    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tmp-dir\") pod \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\" (UID: \"19fdce9d-7230-4a5d-82eb-2eb3cbf08049\") "
Jul 16 23:12:23.336050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335883    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-home" (OuterVolumeSpecName: "home") pod "19fdce9d-7230-4a5d-82eb-2eb3cbf08049" (UID: "19fdce9d-7230-4a5d-82eb-2eb3cbf08049"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:23.336050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.335987    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-model-cache" (OuterVolumeSpecName: "model-cache") pod "19fdce9d-7230-4a5d-82eb-2eb3cbf08049" (UID: "19fdce9d-7230-4a5d-82eb-2eb3cbf08049"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:23.336050 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.336021    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19fdce9d-7230-4a5d-82eb-2eb3cbf08049" (UID: "19fdce9d-7230-4a5d-82eb-2eb3cbf08049"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:23.336232 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.336174    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-model-cache\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:23.336232 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.336189    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tmp-dir\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:23.336232 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.336197    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-home\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:23.337698 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.337679    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19fdce9d-7230-4a5d-82eb-2eb3cbf08049" (UID: "19fdce9d-7230-4a5d-82eb-2eb3cbf08049"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:12:23.338146 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.338119    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-kube-api-access-lb5zw" (OuterVolumeSpecName: "kube-api-access-lb5zw") pod "19fdce9d-7230-4a5d-82eb-2eb3cbf08049" (UID: "19fdce9d-7230-4a5d-82eb-2eb3cbf08049"). InnerVolumeSpecName "kube-api-access-lb5zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:23.338197 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.338148    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-dshm" (OuterVolumeSpecName: "dshm") pod "19fdce9d-7230-4a5d-82eb-2eb3cbf08049" (UID: "19fdce9d-7230-4a5d-82eb-2eb3cbf08049"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:23.437027 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.436997    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-tls-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:23.437027 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.437024    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb5zw\" (UniqueName: \"kubernetes.io/projected/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-kube-api-access-lb5zw\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:23.437027 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:23.437033    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdce9d-7230-4a5d-82eb-2eb3cbf08049-dshm\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:24.194580 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.194553    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"
Jul 16 23:12:24.195004 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.194590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh" event={"ID":"19fdce9d-7230-4a5d-82eb-2eb3cbf08049","Type":"ContainerDied","Data":"72f629e1610294bab98f4bf9ac5aa9407468c5d13bdd5e4bb970dc89b302090f"}
Jul 16 23:12:24.195004 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.194638    2564 scope.go:117] "RemoveContainer" containerID="682a0780288816ec9e5e9691b87befc19a65730518b3b9e23354043b4ae073cf"
Jul 16 23:12:24.210478 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.210454    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"]
Jul 16 23:12:24.218934 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.217454    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b5738f/rate-test-model-b5738f-kserve-d74d868f5-7kpkh"]
Jul 16 23:12:24.654900 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.654873    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 23:12:24.663400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:24.663377    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2qxqw_d2fa47a0-2e48-42d4-a75d-a1c177d8b6ef/ovn-acl-logging/0.log"
Jul 16 23:12:25.603663 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:25.603632    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19fdce9d-7230-4a5d-82eb-2eb3cbf08049" path="/var/lib/kubelet/pods/19fdce9d-7230-4a5d-82eb-2eb3cbf08049/volumes"
Jul 16 23:12:26.596255 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:26.596223    2564 scope.go:117] "RemoveContainer" containerID="ef84a3d47e1785ece0bbf8eebce39f333a545bc6f7a6640093f7b3c3c79a2f66"
Jul 16 23:12:27.213743 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:27.213713    2564 generic.go:356] "Generic (PLEG): container finished" podID="11666424-0504-402f-ac84-8c2d6cce9712" containerID="7b257bbc88aae1268eef653872cd0f6fb31d38e04576c57dff2f69f63d7d49db" exitCode=6
Jul 16 23:12:27.214181 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:27.213762    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" event={"ID":"11666424-0504-402f-ac84-8c2d6cce9712","Type":"ContainerDied","Data":"7b257bbc88aae1268eef653872cd0f6fb31d38e04576c57dff2f69f63d7d49db"}
Jul 16 23:12:27.214181 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:27.213825    2564 scope.go:117] "RemoveContainer" containerID="ef84a3d47e1785ece0bbf8eebce39f333a545bc6f7a6640093f7b3c3c79a2f66"
Jul 16 23:12:27.214181 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:27.214175    2564 scope.go:117] "RemoveContainer" containerID="7b257bbc88aae1268eef653872cd0f6fb31d38e04576c57dff2f69f63d7d49db"
Jul 16 23:12:27.214437 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:12:27.214416    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-7s6vs_odh-ai-gateway-infra(11666424-0504-402f-ac84-8c2d6cce9712)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" podUID="11666424-0504-402f-ac84-8c2d6cce9712"
Jul 16 23:12:28.237902 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:28.237862    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"]
Jul 16 23:12:28.371003 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:28.370976    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:28.486886 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:28.486853    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11666424-0504-402f-ac84-8c2d6cce9712-maas-api-service-ca\") pod \"11666424-0504-402f-ac84-8c2d6cce9712\" (UID: \"11666424-0504-402f-ac84-8c2d6cce9712\") "
Jul 16 23:12:28.487251 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:28.487227    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11666424-0504-402f-ac84-8c2d6cce9712-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11666424-0504-402f-ac84-8c2d6cce9712" (UID: "11666424-0504-402f-ac84-8c2d6cce9712"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:12:28.588203 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:28.588107    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11666424-0504-402f-ac84-8c2d6cce9712-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:29.227365 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:29.227339    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"
Jul 16 23:12:29.227571 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:29.227333    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs" event={"ID":"11666424-0504-402f-ac84-8c2d6cce9712","Type":"ContainerDied","Data":"b5d709aa5aa6cf0ffce757294819408a91834648a37d3ab47cd4e22aae4ee0c5"}
Jul 16 23:12:29.227571 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:29.227492    2564 scope.go:117] "RemoveContainer" containerID="7b257bbc88aae1268eef653872cd0f6fb31d38e04576c57dff2f69f63d7d49db"
Jul 16 23:12:29.250581 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:29.250549    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"]
Jul 16 23:12:29.257277 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:29.257252    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7s6vs"]
Jul 16 23:12:29.602251 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:29.602169    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11666424-0504-402f-ac84-8c2d6cce9712" path="/var/lib/kubelet/pods/11666424-0504-402f-ac84-8c2d6cce9712/volumes"
Jul 16 23:12:42.372103 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:42.372065    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:12:53.728127 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:53.728047    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"]
Jul 16 23:12:53.728552 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:53.728301    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy" containerID="cri-o://2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4" gracePeriod=30
Jul 16 23:12:54.077875 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.077789    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"]
Jul 16 23:12:54.078150 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078122    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19fdce9d-7230-4a5d-82eb-2eb3cbf08049" containerName="main"
Jul 16 23:12:54.078150 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078148    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19fdce9d-7230-4a5d-82eb-2eb3cbf08049" containerName="main"
Jul 16 23:12:54.078253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078168    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078174    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078182    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078188    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078202    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:54.078253 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078208    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:54.078433 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078260    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19fdce9d-7230-4a5d-82eb-2eb3cbf08049" containerName="main"
Jul 16 23:12:54.078433 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078270    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078433 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078276    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078433 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078283    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:54.078433 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.078289    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55fa44d3-622a-49b3-b39b-dbaee30813aa" containerName="revoke-keys"
Jul 16 23:12:54.081319 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.081298    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:54.083620 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.083598    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:12:54.083752 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.083688    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:12:54.088113 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.088092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"]
Jul 16 23:12:54.117694 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.117663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/546f17d3-e6dc-4f7b-9485-19be43ab6aeb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf\" (UID: \"546f17d3-e6dc-4f7b-9485-19be43ab6aeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:54.218969 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.218927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/546f17d3-e6dc-4f7b-9485-19be43ab6aeb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf\" (UID: \"546f17d3-e6dc-4f7b-9485-19be43ab6aeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:54.219581 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.219561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/546f17d3-e6dc-4f7b-9485-19be43ab6aeb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf\" (UID: \"546f17d3-e6dc-4f7b-9485-19be43ab6aeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:54.392870 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.392786    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:54.524536 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:54.524509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"]
Jul 16 23:12:54.525981 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:12:54.525956    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546f17d3_e6dc_4f7b_9485_19be43ab6aeb.slice/crio-1bd4888cd7e238e60a68bc03963908ce09da955588b7e5335a646f566336bca1 WatchSource:0}: Error finding container 1bd4888cd7e238e60a68bc03963908ce09da955588b7e5335a646f566336bca1: Status 404 returned error can't find the container with id 1bd4888cd7e238e60a68bc03963908ce09da955588b7e5335a646f566336bca1
Jul 16 23:12:55.366826 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:55.366727    2564 generic.go:356] "Generic (PLEG): container finished" podID="546f17d3-e6dc-4f7b-9485-19be43ab6aeb" containerID="02e64f752921fb4fe02ab20a6b612a2d5c288d17ac6db150ff785e07e8c7155e" exitCode=0
Jul 16 23:12:55.367203 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:55.366812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf" event={"ID":"546f17d3-e6dc-4f7b-9485-19be43ab6aeb","Type":"ContainerDied","Data":"02e64f752921fb4fe02ab20a6b612a2d5c288d17ac6db150ff785e07e8c7155e"}
Jul 16 23:12:55.367203 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:55.366882    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf" event={"ID":"546f17d3-e6dc-4f7b-9485-19be43ab6aeb","Type":"ContainerStarted","Data":"1bd4888cd7e238e60a68bc03963908ce09da955588b7e5335a646f566336bca1"}
Jul 16 23:12:56.508862 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:56.508837    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:56.542739 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:56.542705    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/546f17d3-e6dc-4f7b-9485-19be43ab6aeb-maas-api-service-ca\") pod \"546f17d3-e6dc-4f7b-9485-19be43ab6aeb\" (UID: \"546f17d3-e6dc-4f7b-9485-19be43ab6aeb\") "
Jul 16 23:12:56.543058 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:56.543035    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/546f17d3-e6dc-4f7b-9485-19be43ab6aeb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "546f17d3-e6dc-4f7b-9485-19be43ab6aeb" (UID: "546f17d3-e6dc-4f7b-9485-19be43ab6aeb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:12:56.643848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:56.643772    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/546f17d3-e6dc-4f7b-9485-19be43ab6aeb-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:57.379510 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:57.379479    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf"
Jul 16 23:12:57.379735 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:57.379482    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf" event={"ID":"546f17d3-e6dc-4f7b-9485-19be43ab6aeb","Type":"ContainerDied","Data":"1bd4888cd7e238e60a68bc03963908ce09da955588b7e5335a646f566336bca1"}
Jul 16 23:12:57.379735 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:57.379618    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bd4888cd7e238e60a68bc03963908ce09da955588b7e5335a646f566336bca1"
Jul 16 23:12:58.987563 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:58.987534    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:12:59.065837 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.065744    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-token\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.065837 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.065795    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzvsj\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-kube-api-access-kzvsj\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.065837 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.065839    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-credential-socket\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066124 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.065913    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-podinfo\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066124 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.065958    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istiod-ca-cert\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066124 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.065980    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-envoy\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066124 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066007    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-socket\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066270    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:59.066373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066281    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:59.066373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066317    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:12:59.066373 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-certs\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066570 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066376    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-data\") pod \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\" (UID: \"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c\") "
Jul 16 23:12:59.066570 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066459    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:59.066718 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066694    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.066793 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066723    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.066793 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066736    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.066793 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.066748    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.067224 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.067201    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:59.068209 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.068179    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:59.068209 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.068203    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:12:59.068426 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.068411    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:12:59.068490 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.068465    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-kube-api-access-kzvsj" (OuterVolumeSpecName: "kube-api-access-kzvsj") pod "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" (UID: "ee519fb8-ae9d-47a2-b2ae-9b8cece5951c"). InnerVolumeSpecName "kube-api-access-kzvsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:59.167509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.167458    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.167509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.167503    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzvsj\" (UniqueName: \"kubernetes.io/projected/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-kube-api-access-kzvsj\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.167509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.167515    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.167509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.167523    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.167509 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.167531    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:12:59.392175 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.392074    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerID="2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4" exitCode=0
Jul 16 23:12:59.392175 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.392123    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"
Jul 16 23:12:59.392175 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.392127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" event={"ID":"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c","Type":"ContainerDied","Data":"2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4"}
Jul 16 23:12:59.392400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.392203    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg" event={"ID":"ee519fb8-ae9d-47a2-b2ae-9b8cece5951c","Type":"ContainerDied","Data":"37c02af40e4fc101becd53d68584aee0dc68a45c6abf9a12b814a99db84b833c"}
Jul 16 23:12:59.392400 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.392226    2564 scope.go:117] "RemoveContainer" containerID="2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4"
Jul 16 23:12:59.401413 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.401396    2564 scope.go:117] "RemoveContainer" containerID="8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"
Jul 16 23:12:59.409667 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.409647    2564 scope.go:117] "RemoveContainer" containerID="2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4"
Jul 16 23:12:59.409948 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:12:59.409925    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4\": container with ID starting with 2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4 not found: ID does not exist" containerID="2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4"
Jul 16 23:12:59.410025 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.409956    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4"} err="failed to get container status \"2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4\": rpc error: code = NotFound desc = could not find container \"2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4\": container with ID starting with 2502451e853ca3bbad1dfd223bba593984f70e1fb72c1f7c967d61d6f4df24c4 not found: ID does not exist"
Jul 16 23:12:59.410025 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.409973    2564 scope.go:117] "RemoveContainer" containerID="8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"
Jul 16 23:12:59.410238 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:12:59.410218    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0\": container with ID starting with 8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0 not found: ID does not exist" containerID="8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"
Jul 16 23:12:59.410291 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.410245    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0"} err="failed to get container status \"8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0\": rpc error: code = NotFound desc = could not find container \"8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0\": container with ID starting with 8186463e8c8f2fb057d52d45c396ab0ec7057ce92f8fd68ebdf36c2a7615cad0 not found: ID does not exist"
Jul 16 23:12:59.413691 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.413667    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"]
Jul 16 23:12:59.419088 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.419064    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b5738f-openshift-default-5bc8f9bd5f-55mzg"]
Jul 16 23:12:59.495595 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495560    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"]
Jul 16 23:12:59.495910 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495898    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="546f17d3-e6dc-4f7b-9485-19be43ab6aeb" containerName="revoke-keys"
Jul 16 23:12:59.495963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495911    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="546f17d3-e6dc-4f7b-9485-19be43ab6aeb" containerName="revoke-keys"
Jul 16 23:12:59.495963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495921    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:59.495963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495926    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11666424-0504-402f-ac84-8c2d6cce9712" containerName="revoke-keys"
Jul 16 23:12:59.495963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495933    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy"
Jul 16 23:12:59.495963 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495938    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy"
Jul 16 23:12:59.496151 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.495997    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy"
Jul 16 23:12:59.496151 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.496006    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="546f17d3-e6dc-4f7b-9485-19be43ab6aeb" containerName="revoke-keys"
Jul 16 23:12:59.500406 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.500389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:12:59.503006 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.502983    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:12:59.503263 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.503012    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:12:59.506584 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.506561    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"]
Jul 16 23:12:59.570531 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.570502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9eb064-8f0b-4c6e-932d-7d09f69d96df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-7l8ns\" (UID: \"5f9eb064-8f0b-4c6e-932d-7d09f69d96df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:12:59.607577 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.607543    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" path="/var/lib/kubelet/pods/ee519fb8-ae9d-47a2-b2ae-9b8cece5951c/volumes"
Jul 16 23:12:59.671617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.671525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9eb064-8f0b-4c6e-932d-7d09f69d96df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-7l8ns\" (UID: \"5f9eb064-8f0b-4c6e-932d-7d09f69d96df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:12:59.672196 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.672175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9eb064-8f0b-4c6e-932d-7d09f69d96df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-234edc78-7l8ns\" (UID: \"5f9eb064-8f0b-4c6e-932d-7d09f69d96df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:12:59.812516 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.812477    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:12:59.957466 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:12:59.957440    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"]
Jul 16 23:12:59.959283 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:12:59.959246    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9eb064_8f0b_4c6e_932d_7d09f69d96df.slice/crio-1e51afe82fa8f22ef4a83d8bfcb1353b8efea5cefa3057c489e906923d8aa685 WatchSource:0}: Error finding container 1e51afe82fa8f22ef4a83d8bfcb1353b8efea5cefa3057c489e906923d8aa685: Status 404 returned error can't find the container with id 1e51afe82fa8f22ef4a83d8bfcb1353b8efea5cefa3057c489e906923d8aa685
Jul 16 23:13:00.400153 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:00.400090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" event={"ID":"5f9eb064-8f0b-4c6e-932d-7d09f69d96df","Type":"ContainerStarted","Data":"5a5bb4eaebe0db2febb03c6e91ce11cbe0f3189153ad113e5820abb9158a1454"}
Jul 16 23:13:00.400604 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:00.400165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" event={"ID":"5f9eb064-8f0b-4c6e-932d-7d09f69d96df","Type":"ContainerStarted","Data":"1e51afe82fa8f22ef4a83d8bfcb1353b8efea5cefa3057c489e906923d8aa685"}
Jul 16 23:13:00.417898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:00.417851    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" podStartSLOduration=1.417835781 podStartE2EDuration="1.417835781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:13:00.416242909 +0000 UTC m=+3336.736798980" watchObservedRunningTime="2026-07-16 23:13:00.417835781 +0000 UTC m=+3336.738391892"
Jul 16 23:13:01.406796 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:01.406757    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerID="5a5bb4eaebe0db2febb03c6e91ce11cbe0f3189153ad113e5820abb9158a1454" exitCode=6
Jul 16 23:13:01.407229 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:01.406838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" event={"ID":"5f9eb064-8f0b-4c6e-932d-7d09f69d96df","Type":"ContainerDied","Data":"5a5bb4eaebe0db2febb03c6e91ce11cbe0f3189153ad113e5820abb9158a1454"}
Jul 16 23:13:01.407229 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:01.407035    2564 scope.go:117] "RemoveContainer" containerID="5a5bb4eaebe0db2febb03c6e91ce11cbe0f3189153ad113e5820abb9158a1454"
Jul 16 23:13:02.414496 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:02.414398    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerID="74f9409b587df6d45a275bea19615f563a60aa569ad6cff3bdb0c6ed2761b101" exitCode=6
Jul 16 23:13:02.414910 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:02.414491    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" event={"ID":"5f9eb064-8f0b-4c6e-932d-7d09f69d96df","Type":"ContainerDied","Data":"74f9409b587df6d45a275bea19615f563a60aa569ad6cff3bdb0c6ed2761b101"}
Jul 16 23:13:02.414910 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:02.414550    2564 scope.go:117] "RemoveContainer" containerID="5a5bb4eaebe0db2febb03c6e91ce11cbe0f3189153ad113e5820abb9158a1454"
Jul 16 23:13:02.414910 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:02.414642    2564 scope.go:117] "RemoveContainer" containerID="74f9409b587df6d45a275bea19615f563a60aa569ad6cff3bdb0c6ed2761b101"
Jul 16 23:13:02.414910 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:13:02.414891    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-7l8ns_odh-ai-gateway-infra(5f9eb064-8f0b-4c6e-932d-7d09f69d96df)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df"
Jul 16 23:13:03.421237 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:03.421206    2564 scope.go:117] "RemoveContainer" containerID="74f9409b587df6d45a275bea19615f563a60aa569ad6cff3bdb0c6ed2761b101"
Jul 16 23:13:03.421628 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:13:03.421398    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-7l8ns_odh-ai-gateway-infra(5f9eb064-8f0b-4c6e-932d-7d09f69d96df)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df"
Jul 16 23:13:07.477160 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:07.477114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sdqkx"]
Jul 16 23:13:15.601954 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:15.601919    2564 scope.go:117] "RemoveContainer" containerID="74f9409b587df6d45a275bea19615f563a60aa569ad6cff3bdb0c6ed2761b101"
Jul 16 23:13:16.494057 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:16.494023    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerID="893f7a5cd426ef1c21c4dce7352cbcb74661868801371c475265e77db07dd66e" exitCode=6
Jul 16 23:13:16.494251 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:16.494098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" event={"ID":"5f9eb064-8f0b-4c6e-932d-7d09f69d96df","Type":"ContainerDied","Data":"893f7a5cd426ef1c21c4dce7352cbcb74661868801371c475265e77db07dd66e"}
Jul 16 23:13:16.494251 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:16.494177    2564 scope.go:117] "RemoveContainer" containerID="74f9409b587df6d45a275bea19615f563a60aa569ad6cff3bdb0c6ed2761b101"
Jul 16 23:13:16.494546 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:16.494524    2564 scope.go:117] "RemoveContainer" containerID="893f7a5cd426ef1c21c4dce7352cbcb74661868801371c475265e77db07dd66e"
Jul 16 23:13:16.494757 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:13:16.494736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-234edc78-7l8ns_odh-ai-gateway-infra(5f9eb064-8f0b-4c6e-932d-7d09f69d96df)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df"
Jul 16 23:13:17.519085 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:17.519050    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"]
Jul 16 23:13:17.661617 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:17.661592    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:13:17.738086 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:17.738041    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9eb064-8f0b-4c6e-932d-7d09f69d96df-maas-api-service-ca\") pod \"5f9eb064-8f0b-4c6e-932d-7d09f69d96df\" (UID: \"5f9eb064-8f0b-4c6e-932d-7d09f69d96df\") "
Jul 16 23:13:17.738423 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:17.738401    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9eb064-8f0b-4c6e-932d-7d09f69d96df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f9eb064-8f0b-4c6e-932d-7d09f69d96df" (UID: "5f9eb064-8f0b-4c6e-932d-7d09f69d96df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:13:17.839189 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:17.839091    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9eb064-8f0b-4c6e-932d-7d09f69d96df-maas-api-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:18.507557 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:18.507523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns" event={"ID":"5f9eb064-8f0b-4c6e-932d-7d09f69d96df","Type":"ContainerDied","Data":"1e51afe82fa8f22ef4a83d8bfcb1353b8efea5cefa3057c489e906923d8aa685"}
Jul 16 23:13:18.507557 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:18.507551    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"
Jul 16 23:13:18.507784 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:18.507584    2564 scope.go:117] "RemoveContainer" containerID="893f7a5cd426ef1c21c4dce7352cbcb74661868801371c475265e77db07dd66e"
Jul 16 23:13:18.531237 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:18.531209    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"]
Jul 16 23:13:18.538065 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:18.538038    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-234edc78-7l8ns"]
Jul 16 23:13:19.601472 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:19.601429    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" path="/var/lib/kubelet/pods/5f9eb064-8f0b-4c6e-932d-7d09f69d96df/volumes"
Jul 16 23:13:19.804022 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:19.803988    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"]
Jul 16 23:13:19.804352 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:19.804296    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy" containerID="cri-o://344f553850f93883b138492ca6fee3b037f636a0c0abc4c8330966201bdda743" gracePeriod=30
Jul 16 23:13:25.046096 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.046064    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:13:25.139017 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.138915    2564 scope.go:117] "RemoveContainer" containerID="40418b7d521a9ab61956b0e976a31aedfffbe6521d4f22efb0ac3ff536a7ac02"
Jul 16 23:13:25.147687 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.147663    2564 scope.go:117] "RemoveContainer" containerID="344f553850f93883b138492ca6fee3b037f636a0c0abc4c8330966201bdda743"
Jul 16 23:13:25.207929 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.207899    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-credential-socket\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.207944    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-token\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.207960    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435bef17-4ce1-484a-976d-c117d10e9ddb-istiod-ca-cert\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.207979    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-certs\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-socket\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208288 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208099    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-envoy\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208288 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-podinfo\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208288 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208212    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftwxw\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-kube-api-access-ftwxw\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208288 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208245    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-data\") pod \"435bef17-4ce1-484a-976d-c117d10e9ddb\" (UID: \"435bef17-4ce1-484a-976d-c117d10e9ddb\") "
Jul 16 23:13:25.208498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208296    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:13:25.208498 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208451    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:13:25.208610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208575    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-credential-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.208610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208583    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:13:25.208610 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208593    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.208766 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.208651    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/435bef17-4ce1-484a-976d-c117d10e9ddb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:13:25.209128 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.209099    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-data" (OuterVolumeSpecName: "istio-data") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:13:25.210356 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.210325    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-token" (OuterVolumeSpecName: "istio-token") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:13:25.210461 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.210399    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 23:13:25.210672 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.210655    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-kube-api-access-ftwxw" (OuterVolumeSpecName: "kube-api-access-ftwxw") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "kube-api-access-ftwxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:13:25.210862 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.210840    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "435bef17-4ce1-484a-976d-c117d10e9ddb" (UID: "435bef17-4ce1-484a-976d-c117d10e9ddb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:13:25.309347 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309318    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftwxw\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-kube-api-access-ftwxw\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.309347 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309343    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-data\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.309347 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309353    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.309579 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309361    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435bef17-4ce1-484a-976d-c117d10e9ddb-istiod-ca-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.309579 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309369    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-workload-socket\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.309579 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309376    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-envoy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.309579 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.309384    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435bef17-4ce1-484a-976d-c117d10e9ddb-istio-podinfo\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\""
Jul 16 23:13:25.546508 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.546462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" event={"ID":"435bef17-4ce1-484a-976d-c117d10e9ddb","Type":"ContainerDied","Data":"344f553850f93883b138492ca6fee3b037f636a0c0abc4c8330966201bdda743"}
Jul 16 23:13:25.546508 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.546502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"
Jul 16 23:13:25.546732 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.546554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2" event={"ID":"435bef17-4ce1-484a-976d-c117d10e9ddb","Type":"ContainerDied","Data":"aef86aa1db82fca0e9aae9185fa52188ba510a6b336fe33d9781c6b57659ad3f"}
Jul 16 23:13:25.580554 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.580521    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"]
Jul 16 23:13:25.585230 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.585191    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-61f04e-openshift-default-5fc99db7f9-v54h2"]
Jul 16 23:13:25.601348 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:25.601318    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" path="/var/lib/kubelet/pods/435bef17-4ce1-484a-976d-c117d10e9ddb/volumes"
Jul 16 23:13:26.140073 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:26.140040    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt_f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928/manager/0.log"
Jul 16 23:13:26.148388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:26.148348    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt_f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928/copy-manifests/0.log"
Jul 16 23:13:26.868165 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:26.868120    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737350-v7dxq_b9f6ae87-5c45-4260-8bb7-031714c32321/cleanup/0.log"
Jul 16 23:13:26.968911 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:26.968883    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737365-2rspn_f575b11b-1bf4-4d8a-a855-299a1a3fecbe/cleanup/0.log"
Jul 16 23:13:27.077563 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:27.077540    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737380-t5cg7_6f9534a6-15a8-4fae-b746-b7c2d39541bf/cleanup/0.log"
Jul 16 23:13:27.397670 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:27.397641    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-bf84b0-7kqpp_63425147-0514-4b6d-9ecf-e28d90b6a908/revoke-keys/0.log"
Jul 16 23:13:27.718274 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:27.718252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-25067845-sxrsd_b336b9dc-0efd-422a-b32e-cc2cd25d3e64/revoke-keys/0.log"
Jul 16 23:13:27.825020 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:27.824992    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-bd998308-jm4tn_5395fdc1-814a-44a5-a85c-a3af4d3fa1bf/revoke-keys/0.log"
Jul 16 23:13:28.033003 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:28.032920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-61f04e-chvkf_546f17d3-e6dc-4f7b-9485-19be43ab6aeb/revoke-keys/0.log"
Jul 16 23:13:28.891697 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:28.891666    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj_073f84a5-5a1d-40c1-987e-771ef6d09f3d/util/0.log"
Jul 16 23:13:28.897967 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:28.897923    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj_073f84a5-5a1d-40c1-987e-771ef6d09f3d/pull/0.log"
Jul 16 23:13:28.903725 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:28.903661    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj_073f84a5-5a1d-40c1-987e-771ef6d09f3d/extract/0.log"
Jul 16 23:13:29.008085 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.008056    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7_d560b140-abf1-42b7-bbe9-b907ed8d856e/util/0.log"
Jul 16 23:13:29.014053 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.014032    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7_d560b140-abf1-42b7-bbe9-b907ed8d856e/pull/0.log"
Jul 16 23:13:29.019684 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.019667    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7_d560b140-abf1-42b7-bbe9-b907ed8d856e/extract/0.log"
Jul 16 23:13:29.130173 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.130112    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9_aef2396c-fea2-4fdc-b364-14c63c900547/extract/0.log"
Jul 16 23:13:29.136086 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.136060    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9_aef2396c-fea2-4fdc-b364-14c63c900547/util/0.log"
Jul 16 23:13:29.141972 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.141914    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9_aef2396c-fea2-4fdc-b364-14c63c900547/pull/0.log"
Jul 16 23:13:29.251276 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.251238    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5_d616be2e-4d83-4d91-8640-76078db8c7a4/util/0.log"
Jul 16 23:13:29.257548 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.257520    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5_d616be2e-4d83-4d91-8640-76078db8c7a4/pull/0.log"
Jul 16 23:13:29.263560 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:29.263541    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5_d616be2e-4d83-4d91-8640-76078db8c7a4/extract/0.log"
Jul 16 23:13:30.057070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:30.057042    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sdqkx_a10c4d21-d45a-4c78-a2d2-8b056e30a7ed/limitador/0.log"
Jul 16 23:13:30.516281 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:30.516252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bc6qb_5a704999-554c-449e-8ed3-3490103090e8/istio-proxy/0.log"
Jul 16 23:13:30.638002 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:30.637976    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v29lv_1e9616a3-2269-4532-9c04-b7a8d6c88923/discovery/0.log"
Jul 16 23:13:30.984850 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:30.984818    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/1.log"
Jul 16 23:13:37.380070 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380036    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvjp6/must-gather-d84hd"]
Jul 16 23:13:37.380562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380514    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy"
Jul 16 23:13:37.380562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380533    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy"
Jul 16 23:13:37.380562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380544    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380550    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380563    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy"
Jul 16 23:13:37.380562 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380569    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380589    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380596    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380666    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380678    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380685    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380689    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380697    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee519fb8-ae9d-47a2-b2ae-9b8cece5951c" containerName="istio-proxy"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380753    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380760    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380776    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380782    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9eb064-8f0b-4c6e-932d-7d09f69d96df" containerName="revoke-keys"
Jul 16 23:13:37.380848 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.380828    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="435bef17-4ce1-484a-976d-c117d10e9ddb" containerName="istio-proxy"
Jul 16 23:13:37.383579 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.383560    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.385759 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.385742    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvjp6\"/\"kube-root-ca.crt\""
Jul 16 23:13:37.385898 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.385749    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvjp6\"/\"openshift-service-ca.crt\""
Jul 16 23:13:37.391884 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.391854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvjp6/must-gather-d84hd"]
Jul 16 23:13:37.522647 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.522609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8ccb\" (UniqueName: \"kubernetes.io/projected/7fe11450-80e3-41b0-8548-c16c396678e8-kube-api-access-j8ccb\") pod \"must-gather-d84hd\" (UID: \"7fe11450-80e3-41b0-8548-c16c396678e8\") " pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.522846 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.522665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7fe11450-80e3-41b0-8548-c16c396678e8-must-gather-output\") pod \"must-gather-d84hd\" (UID: \"7fe11450-80e3-41b0-8548-c16c396678e8\") " pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.623866 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.623832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7fe11450-80e3-41b0-8548-c16c396678e8-must-gather-output\") pod \"must-gather-d84hd\" (UID: \"7fe11450-80e3-41b0-8548-c16c396678e8\") " pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.624062 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.623937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8ccb\" (UniqueName: \"kubernetes.io/projected/7fe11450-80e3-41b0-8548-c16c396678e8-kube-api-access-j8ccb\") pod \"must-gather-d84hd\" (UID: \"7fe11450-80e3-41b0-8548-c16c396678e8\") " pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.624213 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.624193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7fe11450-80e3-41b0-8548-c16c396678e8-must-gather-output\") pod \"must-gather-d84hd\" (UID: \"7fe11450-80e3-41b0-8548-c16c396678e8\") " pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.633046 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.632982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8ccb\" (UniqueName: \"kubernetes.io/projected/7fe11450-80e3-41b0-8548-c16c396678e8-kube-api-access-j8ccb\") pod \"must-gather-d84hd\" (UID: \"7fe11450-80e3-41b0-8548-c16c396678e8\") " pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.693620 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.693586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvjp6/must-gather-d84hd"
Jul 16 23:13:37.827344 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:37.827314    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvjp6/must-gather-d84hd"]
Jul 16 23:13:37.828939 ip-10-0-136-7 kubenswrapper[2564]: W0716 23:13:37.828915    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fe11450_80e3_41b0_8548_c16c396678e8.slice/crio-6fd6d36baff3bbeb970c615cbceb383821d44c5847f84ccf1ab5337327179c84 WatchSource:0}: Error finding container 6fd6d36baff3bbeb970c615cbceb383821d44c5847f84ccf1ab5337327179c84: Status 404 returned error can't find the container with id 6fd6d36baff3bbeb970c615cbceb383821d44c5847f84ccf1ab5337327179c84
Jul 16 23:13:38.617494 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:38.617457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/must-gather-d84hd" event={"ID":"7fe11450-80e3-41b0-8548-c16c396678e8","Type":"ContainerStarted","Data":"6fd6d36baff3bbeb970c615cbceb383821d44c5847f84ccf1ab5337327179c84"}
Jul 16 23:13:39.625419 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:39.625378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/must-gather-d84hd" event={"ID":"7fe11450-80e3-41b0-8548-c16c396678e8","Type":"ContainerStarted","Data":"ff2ab258fe02e813a9c24152ce269396137e8fd3f7b2cb1e338c21e7d78d3598"}
Jul 16 23:13:39.625913 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:39.625430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/must-gather-d84hd" event={"ID":"7fe11450-80e3-41b0-8548-c16c396678e8","Type":"ContainerStarted","Data":"2803723fb7ccea0c0eb9dbafeda25e20a8394ecfc95411a9b82dbcec463c1c0a"}
Jul 16 23:13:39.641675 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:39.641619    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvjp6/must-gather-d84hd" podStartSLOduration=1.6921860880000001 podStartE2EDuration="2.641602805s" totalImagesPullingTime="949.416717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:13:39.640742795 +0000 UTC m=+3375.961298868" watchObservedRunningTime="2026-07-16 23:13:39.641602805 +0000 UTC m=+3375.962158877"
Jul 16 23:13:40.346945 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:40.346916    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vxgdd_2ec7c34d-4960-4611-8f14-9f5dac925696/global-pull-secret-syncer/0.log"
Jul 16 23:13:40.442013 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:40.441979    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j27xc_f1aafd55-4f83-4130-834f-0c70c8267793/konnectivity-agent/0.log"
Jul 16 23:13:40.493581 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:40.493549    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-7.ec2.internal_a8daa969aab0eee12d0aa31e5b2b8339/haproxy/0.log"
Jul 16 23:13:44.811342 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:44.811299    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj_073f84a5-5a1d-40c1-987e-771ef6d09f3d/extract/0.log"
Jul 16 23:13:44.858586 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:44.858554    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj_073f84a5-5a1d-40c1-987e-771ef6d09f3d/util/0.log"
Jul 16 23:13:44.883383 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:44.883328    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jkngj_073f84a5-5a1d-40c1-987e-771ef6d09f3d/pull/0.log"
Jul 16 23:13:44.925469 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:44.925386    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7_d560b140-abf1-42b7-bbe9-b907ed8d856e/extract/0.log"
Jul 16 23:13:44.948570 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:44.948472    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7_d560b140-abf1-42b7-bbe9-b907ed8d856e/util/0.log"
Jul 16 23:13:44.975178 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:44.975150    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mb2m7_d560b140-abf1-42b7-bbe9-b907ed8d856e/pull/0.log"
Jul 16 23:13:45.008108 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.008081    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9_aef2396c-fea2-4fdc-b364-14c63c900547/extract/0.log"
Jul 16 23:13:45.034982 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.034955    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9_aef2396c-fea2-4fdc-b364-14c63c900547/util/0.log"
Jul 16 23:13:45.056946 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.056922    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73c48q9_aef2396c-fea2-4fdc-b364-14c63c900547/pull/0.log"
Jul 16 23:13:45.090885 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.090797    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5_d616be2e-4d83-4d91-8640-76078db8c7a4/extract/0.log"
Jul 16 23:13:45.116545 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.116510    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5_d616be2e-4d83-4d91-8640-76078db8c7a4/util/0.log"
Jul 16 23:13:45.137388 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.137360    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b74d5_d616be2e-4d83-4d91-8640-76078db8c7a4/pull/0.log"
Jul 16 23:13:45.822857 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:45.822822    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sdqkx_a10c4d21-d45a-4c78-a2d2-8b056e30a7ed/limitador/0.log"
Jul 16 23:13:47.754674 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.754557    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/alertmanager/0.log"
Jul 16 23:13:47.784570 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.784528    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/config-reloader/0.log"
Jul 16 23:13:47.807271 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.807190    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/kube-rbac-proxy-web/0.log"
Jul 16 23:13:47.833618 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.833589    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/kube-rbac-proxy/0.log"
Jul 16 23:13:47.860696 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.860664    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/kube-rbac-proxy-metric/0.log"
Jul 16 23:13:47.882048 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.882018    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/prom-label-proxy/0.log"
Jul 16 23:13:47.903868 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:47.903839    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f48b920-277d-4cb9-a2bf-53f7dea480c4/init-config-reloader/0.log"
Jul 16 23:13:48.088397 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.088313    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-g7lbc_72744ec4-fd34-49e5-bf0d-666fbb85ae1c/monitoring-plugin/0.log"
Jul 16 23:13:48.116126 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.116103    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5hwsg_db47d412-8c36-4707-9adb-7af234f01ccf/node-exporter/0.log"
Jul 16 23:13:48.144351 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.144325    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5hwsg_db47d412-8c36-4707-9adb-7af234f01ccf/kube-rbac-proxy/0.log"
Jul 16 23:13:48.167705 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.167671    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5hwsg_db47d412-8c36-4707-9adb-7af234f01ccf/init-textfile/0.log"
Jul 16 23:13:48.337858 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.337820    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-kngm7_0a395eff-14dd-4748-908d-e996a653c9e8/kube-rbac-proxy-main/0.log"
Jul 16 23:13:48.361055 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.360979    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-kngm7_0a395eff-14dd-4748-908d-e996a653c9e8/kube-rbac-proxy-self/0.log"
Jul 16 23:13:48.383975 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.383943    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-kngm7_0a395eff-14dd-4748-908d-e996a653c9e8/openshift-state-metrics/0.log"
Jul 16 23:13:48.639245 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.639156    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-67jrk_5ee8dcb5-0a82-4b5e-ab0d-5b4abc415bcc/prometheus-operator-admission-webhook/0.log"
Jul 16 23:13:48.670246 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.670219    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d4d68bd5b-82hds_3ced4254-2433-4a94-8c10-78702f1c1c6c/telemeter-client/0.log"
Jul 16 23:13:48.693740 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.693705    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d4d68bd5b-82hds_3ced4254-2433-4a94-8c10-78702f1c1c6c/reload/0.log"
Jul 16 23:13:48.720318 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:48.720290    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d4d68bd5b-82hds_3ced4254-2433-4a94-8c10-78702f1c1c6c/kube-rbac-proxy/0.log"
Jul 16 23:13:50.776803 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.776766    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"]
Jul 16 23:13:50.784181 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.784129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.787021 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.786995    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wvjp6\"/\"default-dockercfg-sgxhw\""
Jul 16 23:13:50.788534 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.788508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"]
Jul 16 23:13:50.857923 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.857886    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-sys\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.858149 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.857945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-lib-modules\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.858149 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.858025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-proc\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.858149 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.858050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-podres\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.858149 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.858085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7x5v\" (UniqueName: \"kubernetes.io/projected/13294376-6c39-45be-8146-873291ca207a-kube-api-access-m7x5v\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.958888 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.958840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-proc\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.959292 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.959042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-proc\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.959466 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.959260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-podres\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.959771 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.959751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7x5v\" (UniqueName: \"kubernetes.io/projected/13294376-6c39-45be-8146-873291ca207a-kube-api-access-m7x5v\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.960357 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.960335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-sys\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.960583 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.960568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-lib-modules\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.960828 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.959675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-podres\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.960935 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.960758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-lib-modules\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.961022 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.960513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13294376-6c39-45be-8146-873291ca207a-sys\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:50.968505 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:50.968479    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7x5v\" (UniqueName: \"kubernetes.io/projected/13294376-6c39-45be-8146-873291ca207a-kube-api-access-m7x5v\") pod \"perf-node-gather-daemonset-ktlpk\" (UID: \"13294376-6c39-45be-8146-873291ca207a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:51.100601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.100512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:51.181839 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.181792    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-864f587d6b-7bx2q_ab391318-7b5a-4bb0-8f28-6924abda9d89/console/0.log"
Jul 16 23:13:51.266500 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.266088    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"]
Jul 16 23:13:51.722121 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.722083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk" event={"ID":"13294376-6c39-45be-8146-873291ca207a","Type":"ContainerStarted","Data":"7278c53705727d9337e1392ffb128c74116d9ec2006d648b1588b54c7d4b4bfa"}
Jul 16 23:13:51.722121 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.722165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk" event={"ID":"13294376-6c39-45be-8146-873291ca207a","Type":"ContainerStarted","Data":"b8608f23b31bc29f306254a44be0968eb2e1a7f4a84e424bd83b74d79bcdab29"}
Jul 16 23:13:51.722585 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.722552    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:51.740100 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:51.740040    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk" podStartSLOduration=1.740022201 podStartE2EDuration="1.740022201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 23:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:13:51.736736712 +0000 UTC m=+3388.057292787" watchObservedRunningTime="2026-07-16 23:13:51.740022201 +0000 UTC m=+3388.060578275"
Jul 16 23:13:52.947925 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:52.947890    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"]
Jul 16 23:13:52.951877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:52.951826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"
Jul 16 23:13:52.954014 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:52.953994    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:13:52.954112 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:52.954005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:13:52.960160 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:52.960117    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"]
Jul 16 23:13:53.008552 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.008498    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5l475_0a2a6174-b7bb-4201-82d5-8b4b59e930e1/dns/0.log"
Jul 16 23:13:53.026972 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.026939    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5l475_0a2a6174-b7bb-4201-82d5-8b4b59e930e1/kube-rbac-proxy/0.log"
Jul 16 23:13:53.083337 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.083292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa22b5e-afcb-4405-8632-6029cb75a0b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6\" (UID: \"9aa22b5e-afcb-4405-8632-6029cb75a0b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"
Jul 16 23:13:53.096114 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.096084    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2t5nh_cd31ef13-2d2f-45f0-83b1-407feef1f326/dns-node-resolver/0.log"
Jul 16 23:13:53.184779 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.184735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa22b5e-afcb-4405-8632-6029cb75a0b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6\" (UID: \"9aa22b5e-afcb-4405-8632-6029cb75a0b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"
Jul 16 23:13:53.185655 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.185622    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa22b5e-afcb-4405-8632-6029cb75a0b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6\" (UID: \"9aa22b5e-afcb-4405-8632-6029cb75a0b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"
Jul 16 23:13:53.264540 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.264432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"
Jul 16 23:13:53.424716 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.424482    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6"]
Jul 16 23:13:53.717487 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.717448    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67968859f-8gm5q_88f74aca-06f1-43f0-a26c-3daa3e07c667/registry/0.log"
Jul 16 23:13:53.737275 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.737234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6" event={"ID":"9aa22b5e-afcb-4405-8632-6029cb75a0b9","Type":"ContainerStarted","Data":"d56d0fcf3e071868b4bf445d72876e732db480a1d93c930657bfbbb3a4416334"}
Jul 16 23:13:53.792565 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:53.792535    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6tnf_158c8f12-045b-4e96-b870-e71eb26d1365/node-ca/0.log"
Jul 16 23:13:54.722482 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.722451    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bc6qb_5a704999-554c-449e-8ed3-3490103090e8/istio-proxy/0.log"
Jul 16 23:13:54.745647 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.745611    2564 generic.go:356] "Generic (PLEG): container finished" podID="9aa22b5e-afcb-4405-8632-6029cb75a0b9" containerID="e3776bc06db060a4af67bda5d1948acd4254235fef1325dbbb7a39330ae062a7" exitCode=6
Jul 16 23:13:54.745838 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.745694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6" event={"ID":"9aa22b5e-afcb-4405-8632-6029cb75a0b9","Type":"ContainerDied","Data":"e3776bc06db060a4af67bda5d1948acd4254235fef1325dbbb7a39330ae062a7"}
Jul 16 23:13:54.745838 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.745825    2564 scope.go:117] "RemoveContainer" containerID="e3776bc06db060a4af67bda5d1948acd4254235fef1325dbbb7a39330ae062a7"
Jul 16 23:13:54.873601 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.873548    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v29lv_1e9616a3-2269-4532-9c04-b7a8d6c88923/discovery/0.log"
Jul 16 23:13:54.955613 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.955581    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/0.log"
Jul 16 23:13:54.998261 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:54.998236    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/1.log"
Jul 16 23:13:55.756016 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:55.755914    2564 generic.go:356] "Generic (PLEG): container finished" podID="9aa22b5e-afcb-4405-8632-6029cb75a0b9" containerID="22540ece9146c7660978861dfc2a19446b5d064ea6192ae04de46cb2c33878a0" exitCode=6
Jul 16 23:13:55.756016 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:55.756003    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6" event={"ID":"9aa22b5e-afcb-4405-8632-6029cb75a0b9","Type":"ContainerDied","Data":"22540ece9146c7660978861dfc2a19446b5d064ea6192ae04de46cb2c33878a0"}
Jul 16 23:13:55.756543 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:55.756060    2564 scope.go:117] "RemoveContainer" containerID="e3776bc06db060a4af67bda5d1948acd4254235fef1325dbbb7a39330ae062a7"
Jul 16 23:13:55.756543 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:55.756407    2564 scope.go:117] "RemoveContainer" containerID="22540ece9146c7660978861dfc2a19446b5d064ea6192ae04de46cb2c33878a0"
Jul 16 23:13:55.756694 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:13:55.756673    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6_odh-ai-gateway-infra(9aa22b5e-afcb-4405-8632-6029cb75a0b9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6" podUID="9aa22b5e-afcb-4405-8632-6029cb75a0b9"
Jul 16 23:13:55.816741 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:55.816714    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q6g7l_7b8ce522-ef9d-4655-89a6-6e96b2fb0f12/serve-healthcheck-canary/0.log"
Jul 16 23:13:56.465543 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:56.465517    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmz6v_8e76aeb7-7a77-4efa-8798-780abcdf5801/kube-rbac-proxy/0.log"
Jul 16 23:13:56.486015 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:56.485984    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmz6v_8e76aeb7-7a77-4efa-8798-780abcdf5801/exporter/0.log"
Jul 16 23:13:56.507626 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:56.507590    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmz6v_8e76aeb7-7a77-4efa-8798-780abcdf5801/extractor/0.log"
Jul 16 23:13:56.765812 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:56.765694    2564 scope.go:117] "RemoveContainer" containerID="22540ece9146c7660978861dfc2a19446b5d064ea6192ae04de46cb2c33878a0"
Jul 16 23:13:56.766288 ip-10-0-136-7 kubenswrapper[2564]: E0716 23:13:56.766045    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6_odh-ai-gateway-infra(9aa22b5e-afcb-4405-8632-6029cb75a0b9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5cb4a56a-42kv6" podUID="9aa22b5e-afcb-4405-8632-6029cb75a0b9"
Jul 16 23:13:57.741159 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:57.741104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-ktlpk"
Jul 16 23:13:59.047742 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:59.047718    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt_f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928/manager/0.log"
Jul 16 23:13:59.072203 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:13:59.072177    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-4s7pt_f67795cc-3b7b-4ccb-8aa7-80a5b5cbf928/copy-manifests/0.log"
Jul 16 23:14:00.680238 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:00.680150    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67c6655946-gbvs7_cc547ee2-89a9-47ab-ab66-1132977e07a8/manager/0.log"
Jul 16 23:14:01.199590 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:01.199562    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bc6qb_5a704999-554c-449e-8ed3-3490103090e8/istio-proxy/0.log"
Jul 16 23:14:01.337210 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:01.337176    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v29lv_1e9616a3-2269-4532-9c04-b7a8d6c88923/discovery/0.log"
Jul 16 23:14:01.378989 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:01.378957    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/0.log"
Jul 16 23:14:01.416877 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:01.416842    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/1.log"
Jul 16 23:14:02.662892 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:02.662800    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bc6qb_5a704999-554c-449e-8ed3-3490103090e8/istio-proxy/0.log"
Jul 16 23:14:02.806328 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:02.806304    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v29lv_1e9616a3-2269-4532-9c04-b7a8d6c88923/discovery/0.log"
Jul 16 23:14:02.859614 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:02.859585    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/0.log"
Jul 16 23:14:02.901965 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:02.901929    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mhh8v_328a244c-177f-4f43-9187-8338e3e80b73/istio-proxy/1.log"
Jul 16 23:14:07.216743 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.216714    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/kube-multus-additional-cni-plugins/0.log"
Jul 16 23:14:07.237481 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.237457    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/egress-router-binary-copy/0.log"
Jul 16 23:14:07.256802 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.256775    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/cni-plugins/0.log"
Jul 16 23:14:07.277222 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.277196    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/bond-cni-plugin/0.log"
Jul 16 23:14:07.297678 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.297652    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/routeoverride-cni/0.log"
Jul 16 23:14:07.318763 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.318736    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/whereabouts-cni-bincopy/0.log"
Jul 16 23:14:07.340153 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.340117    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kppsg_f584b484-08cd-4488-8683-954e160667c4/whereabouts-cni/0.log"
Jul 16 23:14:07.628247 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.628158    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kf5qd_7b1e5304-a728-4a7d-8627-e0f909844590/kube-multus/0.log"
Jul 16 23:14:07.753227 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.753039    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pkgx8_68c5002e-be2b-48ff-9d4d-8a36d913f6b0/network-metrics-daemon/0.log"
Jul 16 23:14:07.769246 ip-10-0-136-7 kubenswrapper[2564]: I0716 23:14:07.769218    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pkgx8_68c5002e-be2b-48ff-9d4d-8a36d913f6b0/kube-rbac-proxy/0.log"