Jul 18 00:40:19.004799 ip-10-0-135-55 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 00:40:19.367555 ip-10-0-135-55 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 00:40:19.367555 ip-10-0-135-55 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 00:40:19.367555 ip-10-0-135-55 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 00:40:19.367555 ip-10-0-135-55 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 00:40:19.367555 ip-10-0-135-55 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 00:40:19.369298 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.369205    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 00:40:19.374777 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374755    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 00:40:19.374777 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374777    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374781    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374784    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374787    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374791    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374794    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374797    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374800    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374803    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374805    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374809    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374812    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374815    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374818    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374820    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374823    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374826    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374828    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374831    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 00:40:19.374849 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374833    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374836    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374838    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374841    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374844    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374846    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374849    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374852    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374855    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374858    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374861    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374864    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374867    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374870    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374873    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374875    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374877    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374880    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374890    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374893    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 00:40:19.375303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374895    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374899    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374902    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374905    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374907    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374910    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374912    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374915    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374917    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374928    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374930    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374933    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374936    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374939    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374941    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374944    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374947    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374949    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374952    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374955    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 00:40:19.375832 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374958    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374961    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374965    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374969    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374971    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374974    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374976    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374979    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374981    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374984    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374986    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374989    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374991    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374994    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.374997    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375000    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375002    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375005    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375007    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375009    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 00:40:19.376314 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375012    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375014    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375017    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375022    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375026    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375028    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375032    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375035    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375038    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375042    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375044    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375047    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375050    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375053    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375056    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375059    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375062    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375064    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375067    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 00:40:19.376816 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375833    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375840    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375844    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375848    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375850    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375853    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375856    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375859    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375862    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375865    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375867    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375870    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375873    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375875    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375878    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375880    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375882    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375885    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375887    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375890    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 00:40:19.377368 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375892    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375895    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375898    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375900    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375903    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375906    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375910    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375912    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375915    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375917    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375920    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375922    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375925    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375927    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375929    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375933    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375937    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375940    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375942    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 00:40:19.377878 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375945    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375948    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375952    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375955    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375957    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375960    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375963    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375965    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375968    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375971    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375973    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375976    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375979    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375982    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375984    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375988    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375991    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375994    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.375997    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 00:40:19.378343 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376000    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376003    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376005    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376008    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376010    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376013    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376015    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376017    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376020    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376022    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376024    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376027    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376029    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376032    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376036    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376040    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376043    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376045    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376048    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376050    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 00:40:19.378815 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376053    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376055    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376058    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376060    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376063    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376066    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376068    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376070    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376073    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376076    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376078    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376081    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376083    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376086    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376088    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376091    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376093    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376096    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376098    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376101    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 00:40:19.379304 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376103    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376504    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376510    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376512    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376515    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376517    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376520    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376522    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376526    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376529    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376532    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376534    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376536    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376539    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376541    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376544    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376546    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376549    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376552    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376554    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 00:40:19.379817 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376557    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376560    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376563    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376566    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376568    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376571    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376574    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376576    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376579    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376582    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376584    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376586    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376589    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376592    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376594    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376596    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376599    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376601    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376604    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376606    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 00:40:19.380298 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376610    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376616    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376619    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376622    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376624    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376628    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376630    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376633    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376635    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376638    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376640    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376643    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376645    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376648    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376651    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376653    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376656    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376658    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376661    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376663    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 00:40:19.380791 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376666    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376668    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376670    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376673    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376675    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376678    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376680    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376682    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376685    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376687    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376690    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376692    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376695    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376700    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376703    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376706    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376708    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376711    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376713    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 00:40:19.381303 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376719    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376722    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376725    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376728    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376731    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376734    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376736    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376738    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376741    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376743    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376746    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376748    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376751    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376754    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376757    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376760    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376763    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376766    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376768    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376771    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 00:40:19.381773 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.376774    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376854    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376862    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376868    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376872    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376876    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376881    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376886    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376891    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376894    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376897    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376901    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376904    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376907    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376913    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376916    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376919    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376922    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376925    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376933    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376936    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376939    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376942    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376946    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 00:40:19.382252 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376949    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376952    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376956    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376959    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376963    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376966    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376969    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376972    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376975    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376979    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376982    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376984    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376987    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376991    2567 flags.go:64] FLAG: --enable-server="true"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376994    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.376999    2567 flags.go:64] FLAG: --event-burst="100"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377002    2567 flags.go:64] FLAG: --event-qps="50"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377005    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377008    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377011    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377015    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377018    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377022    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377039    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377043    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 00:40:19.382879 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377046    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377049    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377052    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377055    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377057    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377060    2567 flags.go:64] FLAG: --feature-gates=""
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377064    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377067    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377070    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377073    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377076    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377079    2567 flags.go:64] FLAG: --help="false"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377082    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377085    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377087    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377090    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377093    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377097    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377099    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377102    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377105    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377110    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377112    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377115    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 00:40:19.383516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377118    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377122    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377125    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377128    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377131    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377135    2567 flags.go:64] FLAG: --lock-file=""
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377137    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377140    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377143    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377149    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377152    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377154    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377157    2567 flags.go:64] FLAG: --logging-format="text"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377160    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377163    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377166    2567 flags.go:64] FLAG: --manifest-url=""
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377169    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377173    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377176    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377180    2567 flags.go:64] FLAG: --max-pods="110"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377183    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377186    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377188    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377192    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377194    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 00:40:19.384172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377197    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377200    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377207    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377210    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377214    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377217    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377220    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377226    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377229    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377232    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377235    2567 flags.go:64] FLAG: --port="10250"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377238    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377241    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09c42901740c21bfe"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377245    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377248    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377251    2567 flags.go:64] FLAG: --register-node="true"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377253    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377257    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377260    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377262    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377265    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377269    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377272    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377275    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377277    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 00:40:19.385165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377280    2567 flags.go:64] FLAG: --runonce="false"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377283    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377286    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377289    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377292    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377294    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377297    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377300    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377303    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377305    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377308    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377312    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377315    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377318    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377321    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377324    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377329    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377332    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377335    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377339    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377343    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377346    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377349    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377352    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377354    2567 flags.go:64] FLAG: --v="2"
Jul 18 00:40:19.385832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377358    2567 flags.go:64] FLAG: --version="false"
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377362    2567 flags.go:64] FLAG: --vmodule=""
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377366    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.377369    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377520    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377524    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377528    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377531    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377534    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377537    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377540    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377543    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377546    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377548    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377551    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377554    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377557    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377560    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377563    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377566    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 00:40:19.386439 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377569    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377571    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377574    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377577    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377579    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377582    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377585    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377589    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377591    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377594    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377596    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377598    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377601    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377603    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377606    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377608    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377611    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377613    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377616    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377619    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 00:40:19.386978 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377621    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377624    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377626    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377629    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377631    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377634    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377636    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377638    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377641    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377645    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377649    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377652    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377654    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377657    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377660    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377662    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377665    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377667    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377670    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377674    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 00:40:19.387465 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377676    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377679    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377681    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377683    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377686    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377688    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377691    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377693    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377696    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377699    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377701    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377704    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377706    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377708    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377711    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377713    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377717    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377721    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377723    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 00:40:19.388023 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377726    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377728    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377731    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377734    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377737    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377739    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377742    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377744    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377747    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377749    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377752    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377754    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377760    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377763    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377766    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377769    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377772    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377774    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377776    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 00:40:19.388477 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377779    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377781    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377784    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377786    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.377789    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.378394    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.386787    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.386806    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.387578    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.387586    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387676    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387682    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387685    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387688    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 00:40:19.388954 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387691    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387694    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387697    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387700    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387702    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387705    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387707    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387710    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387712    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387715    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387718    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387720    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387722    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387725    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387728    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387731    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387733    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387736    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387743    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387746    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 00:40:19.389292 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387748    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387751    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387753    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387756    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387759    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387761    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387764    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387767    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387770    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387772    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387774    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387777    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387779    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387782    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387786    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387789    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387792    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387795    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387799    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 00:40:19.389779 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387802    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387806    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387810    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387814    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387817    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387820    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387823    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387826    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387829    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387831    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387833    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387836    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387838    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387841    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387844    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387846    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387849    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387852    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387855    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 00:40:19.390245 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387858    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387860    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387863    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387865    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387868    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387870    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387873    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387875    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387877    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387880    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387883    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387885    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387887    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387890    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387893    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387895    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387898    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387901    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387904    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387906    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 00:40:19.390718 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387909    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387911    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387914    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387917    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387919    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387922    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387924    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387927    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387929    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387932    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387934    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387937    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387940    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387942    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387945    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387947    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 00:40:19.391205 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.387949    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.387954    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388082    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388087    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388090    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388093    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388098    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388102    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388105    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388108    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388111    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388113    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388116    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388118    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 00:40:19.391692 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388121    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388124    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388127    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388130    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388132    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388135    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388138    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388141    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388143    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388145    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388148    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388151    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388153    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388156    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388159    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388161    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388164    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388167    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388169    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388172    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 00:40:19.392028 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388174    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388176    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388179    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388181    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388184    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388186    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388189    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388192    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388194    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388196    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388199    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388202    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388205    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388208    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388211    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388213    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388216    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388219    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388221    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388224    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 00:40:19.392536 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388226    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388229    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388231    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388234    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388236    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388240    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388243    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388246    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388249    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388251    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388254    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388257    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388259    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388262    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388266    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388269    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388271    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388274    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388277    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 00:40:19.393008 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388280    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388282    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388285    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388288    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388290    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388292    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388295    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388298    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388300    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388303    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388306    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388309    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388311    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388314    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388316    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388319    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388321    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388324    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388326    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388329    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 00:40:19.393467 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388331    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388334    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388336    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388339    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388341    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388344    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388346    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:19.388348    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.388353    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.388504    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.390769    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.392183    2567 server.go:1019] "Starting client certificate rotation"
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.392287    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 00:40:19.393952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.392329    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 00:40:19.394599 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.394584    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 00:40:19.396168 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.396152    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 00:40:19.405663 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.405646    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 18 00:40:19.410638 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.410619    2567 log.go:25] "Validated CRI v1 image API"
Jul 18 00:40:19.411506 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.411470    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 00:40:19.414590 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.414572    2567 fs.go:135] Filesystem UUIDs: map[3add296e-69d3-4454-bd04-2168b35435d4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 960856b8-5edd-4f14-9a2d-1c91e87fccc4:/dev/nvme0n1p3]
Jul 18 00:40:19.414636 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.414590    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 00:40:19.418787 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.418671    2567 manager.go:216] Machine: {Timestamp:2026-07-18 00:40:19.418159005 +0000 UTC m=+0.304449669 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100616 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec247b98e872c0a7d84344bfb7078859 SystemUUID:ec247b98-e872-c0a7-d843-44bfb7078859 BootID:b6383afd-5939-47d9-a1f2-f5aa24415856 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:18:7e:5e:95:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:18:7e:5e:95:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:e6:64:6e:3b:92 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 00:40:19.418787 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.418785    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 00:40:19.418910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.418863    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 00:40:19.422532 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.422506    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 00:40:19.423024 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.422814    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 00:40:19.423166 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.423032    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 00:40:19.423772 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.423760    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 00:40:19.423805 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.423775    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 00:40:19.423805 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.423782    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 00:40:19.424407 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.424395    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 00:40:19.426142 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.426132    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 00:40:19.430417 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430404    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 00:40:19.430560 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430550    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 00:40:19.430594 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430581    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 00:40:19.430594 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430593    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 00:40:19.430683 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430609    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 00:40:19.430683 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430618    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 18 00:40:19.430683 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430627    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 00:40:19.430984 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430973    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 00:40:19.431044 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.430992    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 00:40:19.433069 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.433039    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 00:40:19.435153 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.435139    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 00:40:19.435199 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.435171    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 00:40:19.437299 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437283    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437305    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437311    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437319    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437332    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437341    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437352    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 00:40:19.437366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437361    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 00:40:19.437563 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437371    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 00:40:19.437563 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437381    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 00:40:19.437563 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437393    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 00:40:19.437563 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437402    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 00:40:19.437563 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437436    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 00:40:19.437563 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.437445    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 00:40:19.441103 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.441090    2567 server.go:1268] "Started kubelet"
Jul 18 00:40:19.441236 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.441196    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 00:40:19.441346 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.441297    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 00:40:19.441391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.441368    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 00:40:19.441629 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.441611    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 00:40:19.441712 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.441693    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 00:40:19.441761 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.441687    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 00:40:19.443072 ip-10-0-135-55 systemd[1]: Started Kubernetes Kubelet.
Jul 18 00:40:19.443261 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.443242    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 00:40:19.443356 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.443338    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 00:40:19.446530 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.446509    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 00:40:19.449121 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.448284    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18c33b6da300c321  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-07-18 00:40:19.441058593 +0000 UTC m=+0.327349259,LastTimestamp:2026-07-18 00:40:19.441058593 +0000 UTC m=+0.327349259,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Jul 18 00:40:19.451297 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.451280    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 00:40:19.451400 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.451296    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 00:40:19.451457 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.451421    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.451947 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.451927    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 00:40:19.452067 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452050    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 00:40:19.452125 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452064    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 00:40:19.452125 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452079    2567 factory.go:55] Registering systemd factory
Jul 18 00:40:19.452125 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452089    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 18 00:40:19.452125 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452071    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 00:40:19.452309 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452194    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 00:40:19.452309 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452199    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 00:40:19.452476 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452459    2567 factory.go:153] Registering CRI-O factory
Jul 18 00:40:19.452476 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452496    2567 factory.go:223] Registration of the crio container factory successfully
Jul 18 00:40:19.452637 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452524    2567 factory.go:103] Registering Raw factory
Jul 18 00:40:19.452637 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.452537    2567 manager.go:1196] Started watching for new ooms in manager
Jul 18 00:40:19.453093 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.453078    2567 manager.go:349] Starting recovery of all containers
Jul 18 00:40:19.461565 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.456042    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 18 00:40:19.461565 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.456157    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 18 00:40:19.464159 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.463950    2567 manager.go:354] Recovery completed
Jul 18 00:40:19.465944 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.465925    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-czblj"
Jul 18 00:40:19.469223 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.469208    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.471127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471110    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.471192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471144    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.471192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471157    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.471708 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471692    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 00:40:19.471708 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471704    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.471708 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471707    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 00:40:19.471923 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.471725    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 00:40:19.472304 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.472290    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.472362 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.472321    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.472362 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.472336    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.473049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.473035    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-czblj"
Jul 18 00:40:19.473619 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.473550    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-55.ec2.internal.18c33b6da4cb98f6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-55.ec2.internal,UID:ip-10-0-135-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-55.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-55.ec2.internal,},FirstTimestamp:2026-07-18 00:40:19.471128822 +0000 UTC m=+0.357419494,LastTimestamp:2026-07-18 00:40:19.471128822 +0000 UTC m=+0.357419494,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-55.ec2.internal,}"
Jul 18 00:40:19.474157 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.474145    2567 policy_none.go:49] "None policy: Start"
Jul 18 00:40:19.474202 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.474163    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 00:40:19.474202 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.474174    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 00:40:19.476139 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.476129    2567 policy_none.go:47] "Start"
Jul 18 00:40:19.523217 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523195    2567 manager.go:337] "Starting Device Plugin manager"
Jul 18 00:40:19.523330 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.523238    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 00:40:19.523330 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523254    2567 server.go:87] "Starting device plugin registration server"
Jul 18 00:40:19.523584 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523569    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 00:40:19.523659 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523587    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 00:40:19.523702 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523690    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 00:40:19.523790 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523774    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 00:40:19.523790 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.523787    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 00:40:19.524358 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.524342    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 00:40:19.524494 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.524387    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.552066 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.552033    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 00:40:19.557450 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.553632    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 00:40:19.557450 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.553669    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 00:40:19.557450 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.553692    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 00:40:19.557450 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.553731    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 00:40:19.557450 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.555971    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 00:40:19.623982 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.623893    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.624922 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.624900    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.625032 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.624941    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.625032 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.624957    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.625032 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.624987    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.634546 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.634520    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.634639 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.634550    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-55.ec2.internal\": node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.654355 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.654314    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"]
Jul 18 00:40:19.654562 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.654404    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.655855 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.655834    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.655955 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.655869    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.655955 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.655879    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.658382 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.658367    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.658525 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.658509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.658589 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.658549    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.658676 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.658659    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.659120 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.659105    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.659192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.659129    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.659192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.659141    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.659192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.659144    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.659192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.659165    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.659192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.659179    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.661473 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.661456    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.661562 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.661501    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 00:40:19.662191 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.662175    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 00:40:19.662267 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.662210    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 00:40:19.662267 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.662221    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeHasSufficientPID"
Jul 18 00:40:19.684272 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.684248    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-55.ec2.internal\" not found" node="ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.688983 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.688966    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-55.ec2.internal\" not found" node="ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.753738 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.753693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0f668c5d34062566082260ffcecd15d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-55.ec2.internal\" (UID: \"f0f668c5d34062566082260ffcecd15d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.753738 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.753729    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06b460ba2023ffaf7cef4086d61c1653-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"06b460ba2023ffaf7cef4086d61c1653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.753943 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.753754    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06b460ba2023ffaf7cef4086d61c1653-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"06b460ba2023ffaf7cef4086d61c1653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.758745 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.758721    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.854858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.854812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06b460ba2023ffaf7cef4086d61c1653-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"06b460ba2023ffaf7cef4086d61c1653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.854858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.854866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06b460ba2023ffaf7cef4086d61c1653-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"06b460ba2023ffaf7cef4086d61c1653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.855009 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.854876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/06b460ba2023ffaf7cef4086d61c1653-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"06b460ba2023ffaf7cef4086d61c1653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.855009 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.854891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0f668c5d34062566082260ffcecd15d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-55.ec2.internal\" (UID: \"f0f668c5d34062566082260ffcecd15d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.855009 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.854917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0f668c5d34062566082260ffcecd15d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-55.ec2.internal\" (UID: \"f0f668c5d34062566082260ffcecd15d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.855009 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.854962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06b460ba2023ffaf7cef4086d61c1653-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal\" (UID: \"06b460ba2023ffaf7cef4086d61c1653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.858939 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.858918    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.959692 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:19.959666    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:19.987891 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.987865    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:19.993466 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:19.993044    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:20.060600 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:20.060546    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:20.161176 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:20.161133    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:20.261713 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:20.261645    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:21.721108 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:21.721062    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:21.750258 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:21.750226    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 00:40:21.750775 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:21.750434    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 00:40:21.813566 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:21.813536    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 00:40:21.821332 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:21.821305    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:21.836875 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:21.836840    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 00:35:20 +0000 UTC" deadline="2028-02-08 22:36:36.616174085 +0000 UTC"
Jul 18 00:40:21.836875 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:21.836870    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13701h56m14.779307431s"
Jul 18 00:40:21.922086 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:21.922024    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:21.998887 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:21.998839    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f668c5d34062566082260ffcecd15d.slice/crio-c6f9fbfc494501bca67652dd3fc04bda36f8070829b7e53bb2812b3a1561ca16 WatchSource:0}: Error finding container c6f9fbfc494501bca67652dd3fc04bda36f8070829b7e53bb2812b3a1561ca16: Status 404 returned error can't find the container with id c6f9fbfc494501bca67652dd3fc04bda36f8070829b7e53bb2812b3a1561ca16
Jul 18 00:40:21.999401 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:21.999374    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b460ba2023ffaf7cef4086d61c1653.slice/crio-2f27c6bf0b39b95873d49f9137978c5764b54b7b4adfc297df531188d91ce142 WatchSource:0}: Error finding container 2f27c6bf0b39b95873d49f9137978c5764b54b7b4adfc297df531188d91ce142: Status 404 returned error can't find the container with id 2f27c6bf0b39b95873d49f9137978c5764b54b7b4adfc297df531188d91ce142
Jul 18 00:40:22.003418 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.003403    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 00:40:22.022504 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:22.022476    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:22.081039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.081007    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 00:40:22.123091 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:22.123057    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:22.223603 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:22.223529    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-55.ec2.internal\" not found"
Jul 18 00:40:22.279624 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.279597    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 00:40:22.311359 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.311329    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:22.323667 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.323635    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 00:40:22.324668 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.324651    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal"
Jul 18 00:40:22.330692 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.330674    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 00:40:22.691392 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.691353    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 00:40:22.791365 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.791324    2567 apiserver.go:52] "Watching apiserver"
Jul 18 00:40:22.798736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.798703    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 00:40:22.799258 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.799231    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-fwvdq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal","openshift-multus/multus-additional-cni-plugins-smk9b","openshift-multus/multus-hg9rg","openshift-network-operator/iptables-alerter-qqc4k","kube-system/global-pull-secret-syncer-4bl8d","kube-system/konnectivity-agent-8rx49","kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal","openshift-cluster-node-tuning-operator/tuned-f9cmh","openshift-dns/node-resolver-kdrwg","openshift-multus/network-metrics-daemon-qb7rp","openshift-network-diagnostics/network-check-target-45txc","openshift-ovn-kubernetes/ovnkube-node-h9pr9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"]
Jul 18 00:40:22.802100 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.802076    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.804979 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.804847    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 00:40:22.804979 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.804878    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mk9d7\""
Jul 18 00:40:22.805155 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.805066    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 00:40:22.805216 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.805179    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.807446 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.807374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.807926 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.807671    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.807926 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.807839    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hck74\""
Jul 18 00:40:22.808233 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.808214    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.808361 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.808346    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 00:40:22.808493 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.808458    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 00:40:22.808767 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.808751    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 00:40:22.808844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.808771    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 00:40:22.809724 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.809568    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 00:40:22.810583 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.810564    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 00:40:22.810813 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.810797    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 00:40:22.811022 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.811004    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4k68p\""
Jul 18 00:40:22.812224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.811347    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.812224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.811533    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.812224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.811869    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 00:40:22.812519 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.812311    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.814391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.814262    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 00:40:22.814476 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.814399    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8hbc5\""
Jul 18 00:40:22.815094 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.815070    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.817329 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.817306    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.817762 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.817742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wwjr5\""
Jul 18 00:40:22.817896 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.817876    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.817979 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.817949    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 00:40:22.818068 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.818043    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:22.818327 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:22.818121    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:22.820065 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.819910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.822347 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.822327    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.823630 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.823612    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.824143 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.824126    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.825573    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cmbnm\""
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.825828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.825883    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/842c946c-ddda-421a-ac48-d47849301274-konnectivity-ca\") pod \"konnectivity-agent-8rx49\" (UID: \"842c946c-ddda-421a-ac48-d47849301274\") " pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.825922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-systemd-units\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.825950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-cni-netd\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.825982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glkwz\" (UniqueName: \"kubernetes.io/projected/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-kube-api-access-glkwz\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826105    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-cni-multus\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826133    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-kubelet\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-etc-kubernetes\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bea8a24-9778-4af6-b6ea-130307c1a405-host-slash\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-kubelet\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovnkube-config\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovnkube-script-lib\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-os-release\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826435    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-conf-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827049 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826465    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-slash\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-ovn\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-system-cni-dir\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826548    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cnibin\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cni-binary-copy\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-cni-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-run-netns\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826805    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tmh7q\""
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826840    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826932    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.826840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-node-log\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-var-lib-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-cni-bin\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-os-release\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827158    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-cni-bin\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-hostroot\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fddc106e-175c-4243-a732-4ebe482246de-multus-daemon-config\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.827936 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-multus-certs\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgnf\" (UniqueName: \"kubernetes.io/projected/6bea8a24-9778-4af6-b6ea-130307c1a405-kube-api-access-xwgnf\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827344    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovn-node-metrics-cert\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-netns\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdtgm\" (UniqueName: \"kubernetes.io/projected/fddc106e-175c-4243-a732-4ebe482246de-kube-api-access-qdtgm\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827468    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/842c946c-ddda-421a-ac48-d47849301274-agent-certs\") pod \"konnectivity-agent-8rx49\" (UID: \"842c946c-ddda-421a-ac48-d47849301274\") " pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-systemd\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-etc-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827571    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827619    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-log-socket\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827649    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-env-overrides\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827788    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bea8a24-9778-4af6-b6ea-130307c1a405-iptables-alerter-script\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827832    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4tqw\" (UniqueName: \"kubernetes.io/projected/177e03e0-5d99-449f-aac8-09b9b7f8f6db-kube-api-access-j4tqw\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.828808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-system-cni-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-cnibin\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827931    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fddc106e-175c-4243-a732-4ebe482246de-cni-binary-copy\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-socket-dir-parent\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.827993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-k8s-cni-cncf-io\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.828390    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.828858    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.828971    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gnhs4\""
Jul 18 00:40:22.829595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.829347    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.830410 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.830206    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.831544 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.831054    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:22.834579 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.833310    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 00:40:22.836530 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.836500    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-45rfj\""
Jul 18 00:40:22.836723 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.836663    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 00:40:22.837350 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.837327    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:22.837502 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:22.837409    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:22.839680 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.839664    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:22.839761 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:22.839718    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:22.849937 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.849903    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-48g6g"
Jul 18 00:40:22.860473 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.860449    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-48g6g"
Jul 18 00:40:22.912337 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.912308    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 00:40:22.919577 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.919521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal" event={"ID":"f0f668c5d34062566082260ffcecd15d","Type":"ContainerStarted","Data":"c6f9fbfc494501bca67652dd3fc04bda36f8070829b7e53bb2812b3a1561ca16"}
Jul 18 00:40:22.920861 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.920830    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" event={"ID":"06b460ba2023ffaf7cef4086d61c1653","Type":"ContainerStarted","Data":"2f27c6bf0b39b95873d49f9137978c5764b54b7b4adfc297df531188d91ce142"}
Jul 18 00:40:22.929075 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929047    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-run\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.929206 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929101    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/7b946158-2a81-455d-b0f0-68c6df195428-kube-api-access-kdzk6\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.929206 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929130    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ed34f12-7e48-43c6-b50d-bbe40913298b-serviceca\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:22.929206 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929160    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-slash\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929206 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cni-binary-copy\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-cni-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0098140-e04e-4d1c-b440-09c25f6825f9-kubelet-config\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0098140-e04e-4d1c-b440-09c25f6825f9-dbus\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929296    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a8aa0034-92b0-46ce-9347-71f922623496-hosts-file\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929323    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-node-log\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysctl-conf\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-var-lib-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-slash\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929421    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-cni-bin\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-var-lib-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fddc106e-175c-4243-a732-4ebe482246de-multus-daemon-config\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-cni-bin\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-multus-certs\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-node-log\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.929858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929580    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-multus-certs\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.929957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-cni-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cni-binary-copy\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930059    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-device-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930069    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fddc106e-175c-4243-a732-4ebe482246de-multus-daemon-config\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-etc-selinux\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-sys-fs\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.930163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b946158-2a81-455d-b0f0-68c6df195428-etc-tuned\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/842c946c-ddda-421a-ac48-d47849301274-agent-certs\") pod \"konnectivity-agent-8rx49\" (UID: \"842c946c-ddda-421a-ac48-d47849301274\") " pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-systemd\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930338    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-systemd\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-k8s-cni-cncf-io\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930380    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-sys\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-systemd\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930398    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ed34f12-7e48-43c6-b50d-bbe40913298b-host\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/842c946c-ddda-421a-ac48-d47849301274-konnectivity-ca\") pod \"konnectivity-agent-8rx49\" (UID: \"842c946c-ddda-421a-ac48-d47849301274\") " pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-cni-multus\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-kubelet\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-etc-kubernetes\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930544    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js7w8\" (UniqueName: \"kubernetes.io/projected/3aebfac2-b504-44d8-9f4b-46e0e37e191a-kube-api-access-js7w8\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.930571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bea8a24-9778-4af6-b6ea-130307c1a405-host-slash\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930580    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-kubelet\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovnkube-config\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930637    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b946158-2a81-455d-b0f0-68c6df195428-tmp\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930656    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-ovn\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-system-cni-dir\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cnibin\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930755    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xslp\" (UniqueName: \"kubernetes.io/projected/a8aa0034-92b0-46ce-9347-71f922623496-kube-api-access-6xslp\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-modprobe-d\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930796    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-run-netns\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-socket-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930834    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-os-release\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-cni-bin\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-k8s-cni-cncf-io\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930914    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-hostroot\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdtgm\" (UniqueName: \"kubernetes.io/projected/fddc106e-175c-4243-a732-4ebe482246de-kube-api-access-qdtgm\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.930981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-cni-multus\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931088    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bea8a24-9778-4af6-b6ea-130307c1a405-host-slash\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-cni-bin\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931139    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-ovn\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931161    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-kubelet\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-hostroot\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-var-lib-kubelet\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931208    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-system-cni-dir\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8aa0034-92b0-46ce-9347-71f922623496-tmp-dir\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931235    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-cnibin\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgnf\" (UniqueName: \"kubernetes.io/projected/6bea8a24-9778-4af6-b6ea-130307c1a405-kube-api-access-xwgnf\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovn-node-metrics-cert\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-os-release\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-netns\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/842c946c-ddda-421a-ac48-d47849301274-konnectivity-ca\") pod \"konnectivity-agent-8rx49\" (UID: \"842c946c-ddda-421a-ac48-d47849301274\") " pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-kubernetes\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.931974 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysctl-d\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-var-lib-kubelet\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26pp\" (UniqueName: \"kubernetes.io/projected/0ed34f12-7e48-43c6-b50d-bbe40913298b-kube-api-access-r26pp\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-etc-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-etc-kubernetes\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovnkube-config\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-log-socket\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-run-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931819    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931820    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-env-overrides\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931817    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-run-netns\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-log-socket\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4tqw\" (UniqueName: \"kubernetes.io/projected/177e03e0-5d99-449f-aac8-09b9b7f8f6db-kube-api-access-j4tqw\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931897    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-etc-openvswitch\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.932844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-cnibin\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.931983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fddc106e-175c-4243-a732-4ebe482246de-cni-binary-copy\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bea8a24-9778-4af6-b6ea-130307c1a405-iptables-alerter-script\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-env-overrides\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932177    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-host-run-netns\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932232    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-cnibin\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-system-cni-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932335    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-socket-dir-parent\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-system-cni-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysconfig\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932426    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-socket-dir-parent\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-lib-modules\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-host\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-systemd-units\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.933714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-cni-netd\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glkwz\" (UniqueName: \"kubernetes.io/projected/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-kube-api-access-glkwz\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fddc106e-175c-4243-a732-4ebe482246de-cni-binary-copy\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932638    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-cni-netd\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932638    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-systemd-units\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bea8a24-9778-4af6-b6ea-130307c1a405-iptables-alerter-script\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932666    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-registration-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgv95\" (UniqueName: \"kubernetes.io/projected/984d0621-5433-4613-b1ff-41f803ca0a53-kube-api-access-cgv95\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932733    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovnkube-script-lib\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-os-release\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-conf-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/177e03e0-5d99-449f-aac8-09b9b7f8f6db-os-release\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.932984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fddc106e-175c-4243-a732-4ebe482246de-multus-conf-dir\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.934545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.933294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovnkube-script-lib\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.935119 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.934561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/842c946c-ddda-421a-ac48-d47849301274-agent-certs\") pod \"konnectivity-agent-8rx49\" (UID: \"842c946c-ddda-421a-ac48-d47849301274\") " pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:22.936009 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.935986    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-ovn-node-metrics-cert\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:22.942890 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.942826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgnf\" (UniqueName: \"kubernetes.io/projected/6bea8a24-9778-4af6-b6ea-130307c1a405-kube-api-access-xwgnf\") pod \"iptables-alerter-qqc4k\" (UID: \"6bea8a24-9778-4af6-b6ea-130307c1a405\") " pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:22.943440 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.943418    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdtgm\" (UniqueName: \"kubernetes.io/projected/fddc106e-175c-4243-a732-4ebe482246de-kube-api-access-qdtgm\") pod \"multus-hg9rg\" (UID: \"fddc106e-175c-4243-a732-4ebe482246de\") " pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:22.943888 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.943868    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4tqw\" (UniqueName: \"kubernetes.io/projected/177e03e0-5d99-449f-aac8-09b9b7f8f6db-kube-api-access-j4tqw\") pod \"multus-additional-cni-plugins-smk9b\" (UID: \"177e03e0-5d99-449f-aac8-09b9b7f8f6db\") " pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:22.944484 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:22.944468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glkwz\" (UniqueName: \"kubernetes.io/projected/b5e126a6-ba94-416e-8638-a4a9ecf7b93f-kube-api-access-glkwz\") pod \"ovnkube-node-h9pr9\" (UID: \"b5e126a6-ba94-416e-8638-a4a9ecf7b93f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:23.033322 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8aa0034-92b0-46ce-9347-71f922623496-tmp-dir\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.033322 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-kubernetes\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033569 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysctl-d\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033569 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysctl-d\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033569 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033554    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-kubernetes\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-var-lib-kubelet\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26pp\" (UniqueName: \"kubernetes.io/projected/0ed34f12-7e48-43c6-b50d-bbe40913298b-kube-api-access-r26pp\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysconfig\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-lib-modules\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-host\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8aa0034-92b0-46ce-9347-71f922623496-tmp-dir\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.033911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-host\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-registration-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.033911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033805    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-var-lib-kubelet\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.033911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-registration-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034073 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.033918    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034073 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.034013    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:23.034073 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034031    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysconfig\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034175 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.034091    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:23.534068906 +0000 UTC m=+3.061573867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:23.034175 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034117    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-lib-modules\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034175 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgv95\" (UniqueName: \"kubernetes.io/projected/984d0621-5433-4613-b1ff-41f803ca0a53-kube-api-access-cgv95\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:23.034175 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-run\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/7b946158-2a81-455d-b0f0-68c6df195428-kube-api-access-kdzk6\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ed34f12-7e48-43c6-b50d-bbe40913298b-serviceca\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0098140-e04e-4d1c-b440-09c25f6825f9-kubelet-config\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0098140-e04e-4d1c-b440-09c25f6825f9-dbus\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a8aa0034-92b0-46ce-9347-71f922623496-hosts-file\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysctl-conf\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034334 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034308    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-device-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034534 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0098140-e04e-4d1c-b440-09c25f6825f9-kubelet-config\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:23.034534 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034429    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0098140-e04e-4d1c-b440-09c25f6825f9-dbus\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:23.034534 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a8aa0034-92b0-46ce-9347-71f922623496-hosts-file\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.034654 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-sysctl-conf\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034654 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034571    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-run\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034654 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ed34f12-7e48-43c6-b50d-bbe40913298b-serviceca\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.034736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034678    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-device-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034770 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-etc-selinux\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034770 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-sys-fs\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034828 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b946158-2a81-455d-b0f0-68c6df195428-etc-tuned\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034828 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:23.034828 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-systemd\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034915 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-sys-fs\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034915 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-systemd\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034972 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-etc-selinux\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.034972 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034934    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-sys\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.034972 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ed34f12-7e48-43c6-b50d-bbe40913298b-host\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.034972 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js7w8\" (UniqueName: \"kubernetes.io/projected/3aebfac2-b504-44d8-9f4b-46e0e37e191a-kube-api-access-js7w8\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.035082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.034984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b946158-2a81-455d-b0f0-68c6df195428-tmp\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.035082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xslp\" (UniqueName: \"kubernetes.io/projected/a8aa0034-92b0-46ce-9347-71f922623496-kube-api-access-6xslp\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.035082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-modprobe-d\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.035082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035031    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:23.035082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035046    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-socket-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.035304 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3aebfac2-b504-44d8-9f4b-46e0e37e191a-socket-dir\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.035304 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-sys\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.035304 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ed34f12-7e48-43c6-b50d-bbe40913298b-host\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.035454 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.035412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b946158-2a81-455d-b0f0-68c6df195428-etc-modprobe-d\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.035454 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.035475    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:23.035713 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.035520    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:23.535508333 +0000 UTC m=+3.063013272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:23.037919 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.037878    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b946158-2a81-455d-b0f0-68c6df195428-tmp\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.038093 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.038068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b946158-2a81-455d-b0f0-68c6df195428-etc-tuned\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.046403 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.046205    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:23.046403 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.046237    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:23.046403 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.046253    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:23.046403 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.046352    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:23.546332224 +0000 UTC m=+3.073837183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:23.047978 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.047947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgv95\" (UniqueName: \"kubernetes.io/projected/984d0621-5433-4613-b1ff-41f803ca0a53-kube-api-access-cgv95\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:23.048103 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.048012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/7b946158-2a81-455d-b0f0-68c6df195428-kube-api-access-kdzk6\") pod \"tuned-f9cmh\" (UID: \"7b946158-2a81-455d-b0f0-68c6df195428\") " pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.048103 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.048045    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26pp\" (UniqueName: \"kubernetes.io/projected/0ed34f12-7e48-43c6-b50d-bbe40913298b-kube-api-access-r26pp\") pod \"node-ca-fwvdq\" (UID: \"0ed34f12-7e48-43c6-b50d-bbe40913298b\") " pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.048360 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.048334    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js7w8\" (UniqueName: \"kubernetes.io/projected/3aebfac2-b504-44d8-9f4b-46e0e37e191a-kube-api-access-js7w8\") pod \"aws-ebs-csi-driver-node-82d2b\" (UID: \"3aebfac2-b504-44d8-9f4b-46e0e37e191a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.048939 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.048917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xslp\" (UniqueName: \"kubernetes.io/projected/a8aa0034-92b0-46ce-9347-71f922623496-kube-api-access-6xslp\") pod \"node-resolver-kdrwg\" (UID: \"a8aa0034-92b0-46ce-9347-71f922623496\") " pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.117788 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.117751    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:23.126000 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.125976    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:23.141380 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.141356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-smk9b"
Jul 18 00:40:23.149079 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.149059    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hg9rg"
Jul 18 00:40:23.159824 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.159801    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qqc4k"
Jul 18 00:40:23.167794 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.167758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f9cmh"
Jul 18 00:40:23.175610 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.175588    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fwvdq"
Jul 18 00:40:23.183351 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.183323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b"
Jul 18 00:40:23.191029 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.190998    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdrwg"
Jul 18 00:40:23.538616 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.538533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:23.538616 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.538578    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:23.538842 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.538690    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:23.538842 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.538691    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:23.538842 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.538755    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:24.538739078 +0000 UTC m=+4.066244019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:23.538842 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.538772    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:24.53876426 +0000 UTC m=+4.066269197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:23.639559 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.639518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:23.639710 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.639693    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:23.639749 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.639715    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:23.639749 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.639725    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:23.639807 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:23.639778    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:24.639764914 +0000 UTC m=+4.167269852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:23.864255 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.864222    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 00:35:22 +0000 UTC" deadline="2027-12-26 23:49:43.970637235 +0000 UTC"
Jul 18 00:40:23.864255 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:23.864253    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12647h9m20.106388169s"
Jul 18 00:40:24.012047 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:24.012018    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177e03e0_5d99_449f_aac8_09b9b7f8f6db.slice/crio-629326be5388116325e32b92d3e2410c598a13fc88b070690452a2ca516a1deb WatchSource:0}: Error finding container 629326be5388116325e32b92d3e2410c598a13fc88b070690452a2ca516a1deb: Status 404 returned error can't find the container with id 629326be5388116325e32b92d3e2410c598a13fc88b070690452a2ca516a1deb
Jul 18 00:40:24.021974 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:24.021933    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed34f12_7e48_43c6_b50d_bbe40913298b.slice/crio-ee75c702ba8ac9e80cc1361238d97fd769d7221c9e18175d91c0b67c8fa9e98f WatchSource:0}: Error finding container ee75c702ba8ac9e80cc1361238d97fd769d7221c9e18175d91c0b67c8fa9e98f: Status 404 returned error can't find the container with id ee75c702ba8ac9e80cc1361238d97fd769d7221c9e18175d91c0b67c8fa9e98f
Jul 18 00:40:24.022627 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:24.022597    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b946158_2a81_455d_b0f0_68c6df195428.slice/crio-40444fffd8b8010514d77c73c63696cd8eb865924644e86445fbe16168385f99 WatchSource:0}: Error finding container 40444fffd8b8010514d77c73c63696cd8eb865924644e86445fbe16168385f99: Status 404 returned error can't find the container with id 40444fffd8b8010514d77c73c63696cd8eb865924644e86445fbe16168385f99
Jul 18 00:40:24.545312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.544992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:24.545481 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.545187    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:24.545481 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.545352    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:24.545481 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.545405    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:26.545381379 +0000 UTC m=+6.072886331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:24.545481 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.545474    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:24.545696 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.545529    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:26.545513024 +0000 UTC m=+6.073017966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:24.646386 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.645677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:24.646386 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.645894    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:24.646386 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.645918    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:24.646386 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.645932    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:24.646386 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.645992    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:26.645972469 +0000 UTC m=+6.173477420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:24.865133 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.865043    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 00:35:22 +0000 UTC" deadline="2028-02-26 07:55:34.317501642 +0000 UTC"
Jul 18 00:40:24.865133 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.865103    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14119h15m9.452402833s"
Jul 18 00:40:24.914420 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.914392    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:24.914420 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.914412    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:24.914646 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.914525    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:24.914646 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.914564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:24.914757 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.914655    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:24.914757 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:24.914742    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:24.936560 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.936522    2567 generic.go:356] "Generic (PLEG): container finished" podID="06b460ba2023ffaf7cef4086d61c1653" containerID="80416c3fd2fe6f825260df8df814622fbb544665b37b70a247903b8cc91b68d6" exitCode=0
Jul 18 00:40:24.936719 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.936616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" event={"ID":"06b460ba2023ffaf7cef4086d61c1653","Type":"ContainerDied","Data":"80416c3fd2fe6f825260df8df814622fbb544665b37b70a247903b8cc91b68d6"}
Jul 18 00:40:24.945245 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.945203    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qqc4k" event={"ID":"6bea8a24-9778-4af6-b6ea-130307c1a405","Type":"ContainerStarted","Data":"0b787c9b98b22c1e0e6aa5aea1694080ba6e6301a2151b1c87e2489f3a8b18a2"}
Jul 18 00:40:24.954714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.954679    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b" event={"ID":"3aebfac2-b504-44d8-9f4b-46e0e37e191a","Type":"ContainerStarted","Data":"1ce5253954d9ae66502941312cf05e2ddb157caeec7fbccad905c8972bd8a5b4"}
Jul 18 00:40:24.961101 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.961069    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hg9rg" event={"ID":"fddc106e-175c-4243-a732-4ebe482246de","Type":"ContainerStarted","Data":"2c8c1bb822f8888bff73b465d3c57c5f201d0638384d93cc7f84c98e79480114"}
Jul 18 00:40:24.979371 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.979334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rx49" event={"ID":"842c946c-ddda-421a-ac48-d47849301274","Type":"ContainerStarted","Data":"c5e03c5cbeec79b1f6b4c73f1b93825c04b3e354174e66ab4c2b783d2e2e8e71"}
Jul 18 00:40:24.999710 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:24.999672    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal" event={"ID":"f0f668c5d34062566082260ffcecd15d","Type":"ContainerStarted","Data":"e065271d5e343c17e42857ebdf7957be4afacf5e15e0d9af49f8e532d3462647"}
Jul 18 00:40:25.014698 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:25.014635    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-55.ec2.internal" podStartSLOduration=3.01461079 podStartE2EDuration="3.01461079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:25.014075847 +0000 UTC m=+4.541580807" watchObservedRunningTime="2026-07-18 00:40:25.01461079 +0000 UTC m=+4.542115751"
Jul 18 00:40:25.014873 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:25.014675    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f9cmh" event={"ID":"7b946158-2a81-455d-b0f0-68c6df195428","Type":"ContainerStarted","Data":"40444fffd8b8010514d77c73c63696cd8eb865924644e86445fbe16168385f99"}
Jul 18 00:40:25.034343 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:25.034303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"46df71fa4cc102e9b74eb52ec4ec24e1c943bc7f7295556a59f2144cbc6f7195"}
Jul 18 00:40:25.037826 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:25.037626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fwvdq" event={"ID":"0ed34f12-7e48-43c6-b50d-bbe40913298b","Type":"ContainerStarted","Data":"ee75c702ba8ac9e80cc1361238d97fd769d7221c9e18175d91c0b67c8fa9e98f"}
Jul 18 00:40:25.058553 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:25.058479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdrwg" event={"ID":"a8aa0034-92b0-46ce-9347-71f922623496","Type":"ContainerStarted","Data":"ad60c221d86b088ddeecbbcccc1d7659a7264a83c8cf4a3b282fa3558744c1b7"}
Jul 18 00:40:25.068229 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:25.068183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerStarted","Data":"629326be5388116325e32b92d3e2410c598a13fc88b070690452a2ca516a1deb"}
Jul 18 00:40:26.088408 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.088257    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" event={"ID":"06b460ba2023ffaf7cef4086d61c1653","Type":"ContainerStarted","Data":"24fb54383ff65cd69193b101786886728894edfeb925e5b6715810bb6bc7ed25"}
Jul 18 00:40:26.102973 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.102866    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-55.ec2.internal" podStartSLOduration=4.102846144 podStartE2EDuration="4.102846144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:26.102700311 +0000 UTC m=+5.630205273" watchObservedRunningTime="2026-07-18 00:40:26.102846144 +0000 UTC m=+5.630351108"
Jul 18 00:40:26.561621 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.561584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:26.561801 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.561704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:26.561899 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.561840    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:26.561950 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.561906    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:30.56188704 +0000 UTC m=+10.089391998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:26.562360 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.562339    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:26.562447 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.562401    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:30.562383879 +0000 UTC m=+10.089888822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:26.663034 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.662994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:26.663222 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.663174    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:26.663222 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.663206    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:26.663222 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.663219    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:26.663426 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.663301    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:30.663263005 +0000 UTC m=+10.190767951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:26.914485 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.914401    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:26.914642 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.914511    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:26.914820 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.914794    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:26.914905 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.914856    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:26.914905 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:26.914896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:26.915007 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:26.914938    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:28.915096 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:28.914404    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:28.915096 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:28.914466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:28.915096 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:28.914478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:28.915096 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:28.914578    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:28.915096 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:28.914703    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:28.915096 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:28.914783    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:30.598196 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:30.598158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:30.598625 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:30.598214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:30.598625 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.598332    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:30.598625 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.598363    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:30.598625 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.598411    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:38.598388859 +0000 UTC m=+18.125893817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:30.598625 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.598431    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:38.598420268 +0000 UTC m=+18.125925209 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:30.699599 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:30.699555    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:30.699768 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.699709    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:30.699768 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.699730    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:30.699768 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.699743    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:30.699909 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.699793    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:38.699780343 +0000 UTC m=+18.227285280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:30.914292 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:30.914196    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:30.914727 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:30.914304    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:30.914727 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.914558    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:30.914727 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:30.914326    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:30.914727 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.914685    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:30.914996 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:30.914780    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:32.914409 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:32.914377    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:32.914877 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:32.914518    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:32.914985 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:32.914954    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:32.915182 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:32.915065    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:32.915255 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:32.915220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:32.915438 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:32.915360    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:34.913340 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:34.913302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:34.913734 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:34.913342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:34.913734 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:34.913413    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:34.913846 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:34.913805    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:34.913881 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:34.913867    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:34.913958 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:34.913939    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:36.913212 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:36.912701    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:36.913212 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:36.912716    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:36.913212 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:36.912846    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:36.913212 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:36.912898    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:36.913212 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:36.913017    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:36.913212 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:36.913097    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:38.668185 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:38.668103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:38.668185 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:38.668146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:38.668620 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.668248    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:38.668620 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.668261    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:38.668620 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.668315    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:54.668301291 +0000 UTC m=+34.195806230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:38.668620 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.668343    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:54.668321102 +0000 UTC m=+34.195826041 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:38.769316 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:38.769247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:38.769483 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.769400    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:38.769483 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.769429    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:38.769483 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.769439    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:38.769576 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.769496    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:54.76948024 +0000 UTC m=+34.296985178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:38.912757 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:38.912723    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:38.912922 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:38.912722    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:38.912922 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.912863    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:38.913046 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.912953    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:38.913046 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:38.912724    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:38.913144 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:38.913047    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:40.913910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:40.913872    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:40.914390 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:40.914001    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:40.914390 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:40.914017    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:40.914390 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:40.914107    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:40.914390 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:40.914140    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:40.914390 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:40.914181    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:42.912757 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:42.912721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:42.912757 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:42.912745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:42.913345 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:42.912745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:42.913345 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:42.912861    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:42.913345 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:42.912978    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:42.913345 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:42.913048    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:43.136415 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:43.135619    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fwvdq" event={"ID":"0ed34f12-7e48-43c6-b50d-bbe40913298b","Type":"ContainerStarted","Data":"5ac2f379d1baf0cfdebd33e8f4542e6a774151430fdcf30fd60bf1ec565b10a2"}
Jul 18 00:40:43.147347 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:43.147314    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hg9rg" event={"ID":"fddc106e-175c-4243-a732-4ebe482246de","Type":"ContainerStarted","Data":"cc202f311c9a61aacefbb43686f5967907ff41efeb13a163569952ceceb21409"}
Jul 18 00:40:43.152627 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:43.152356    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fwvdq" podStartSLOduration=3.214923299 podStartE2EDuration="22.152335215s" totalImagesPullingTime="18.937411916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:43.150192664 +0000 UTC m=+22.677697624" watchObservedRunningTime="2026-07-18 00:40:43.152335215 +0000 UTC m=+22.679840176"
Jul 18 00:40:43.192331 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:43.192100    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hg9rg" podStartSLOduration=3.211508603 podStartE2EDuration="22.19207998s" totalImagesPullingTime="18.980571377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:43.168676362 +0000 UTC m=+22.696181323" watchObservedRunningTime="2026-07-18 00:40:43.19207998 +0000 UTC m=+22.719584943"
Jul 18 00:40:44.152855 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.152603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"2c91368d6c17c534c941b686a24203bbc36f048c78b530c89f81bfee934dd4d6"}
Jul 18 00:40:44.153707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.152952    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"2243ef3d087a7bd9b0f1317213111b64b346ad80561adf871c50a4ea01e67670"}
Jul 18 00:40:44.153707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.153003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"950493fe6c035a269be8df76cead6bb3bcdf2241b8813999b88b464fd85b3756"}
Jul 18 00:40:44.153707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.153044    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"be079f05d6260f760873d457a1ab39ea0b911858dba6cc6f2d20e1f3ee85c24d"}
Jul 18 00:40:44.153707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.153083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"c2cc1562ef1aa51ed3422217feb1f37a64950056c7b8c7fcf9e83c4cb809d7da"}
Jul 18 00:40:44.153707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.153123    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"2d0fdcec5657d2fc6b3da16667a48289e5daa6c12c6b8bd6adde267041ad14e8"}
Jul 18 00:40:44.154812 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.154786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdrwg" event={"ID":"a8aa0034-92b0-46ce-9347-71f922623496","Type":"ContainerStarted","Data":"faa30d3e6b6071d3783fd46646e171d5317a429f0af04b6b73fe12464811f5d6"}
Jul 18 00:40:44.156381 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.156354    2567 generic.go:356] "Generic (PLEG): container finished" podID="177e03e0-5d99-449f-aac8-09b9b7f8f6db" containerID="68609dd9af6c5822a7976d12ccb9f8727bf2943ac86c4b2b065f59eaaffa6750" exitCode=0
Jul 18 00:40:44.156498 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.156431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerDied","Data":"68609dd9af6c5822a7976d12ccb9f8727bf2943ac86c4b2b065f59eaaffa6750"}
Jul 18 00:40:44.158022 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.158004    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b" event={"ID":"3aebfac2-b504-44d8-9f4b-46e0e37e191a","Type":"ContainerStarted","Data":"5c858a3f1bcbc2acacaf145bc51ef96ef97a1c66e5e7ef9bf84ac4c04635ffcc"}
Jul 18 00:40:44.159755 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.159657    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rx49" event={"ID":"842c946c-ddda-421a-ac48-d47849301274","Type":"ContainerStarted","Data":"d95efa7d2ec755939d070cdf6c70eb05f28f86808836c38eb69df4e562f61922"}
Jul 18 00:40:44.161335 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.161310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f9cmh" event={"ID":"7b946158-2a81-455d-b0f0-68c6df195428","Type":"ContainerStarted","Data":"85ceb95fe4693458f8ac991af234364405596bd1b17eb4aacce93a5394daf592"}
Jul 18 00:40:44.169546 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.169501    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kdrwg" podStartSLOduration=4.226086684 podStartE2EDuration="23.169486292s" totalImagesPullingTime="18.943399608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:44.169044882 +0000 UTC m=+23.696549841" watchObservedRunningTime="2026-07-18 00:40:44.169486292 +0000 UTC m=+23.696991252"
Jul 18 00:40:44.182545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.182493    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8rx49" podStartSLOduration=4.248873131 podStartE2EDuration="23.182478533s" totalImagesPullingTime="18.933605402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:44.18213699 +0000 UTC m=+23.709641950" watchObservedRunningTime="2026-07-18 00:40:44.182478533 +0000 UTC m=+23.709983492"
Jul 18 00:40:44.219285 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.219234    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f9cmh" podStartSLOduration=4.27239148 podStartE2EDuration="23.219219212s" totalImagesPullingTime="18.946827732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:44.218794234 +0000 UTC m=+23.746299194" watchObservedRunningTime="2026-07-18 00:40:44.219219212 +0000 UTC m=+23.746724171"
Jul 18 00:40:44.686940 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.686909    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 00:40:44.893965 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.893773    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T00:40:44.686932916Z","UUID":"80de9ab7-d463-47e0-8ecb-fa584bbea284","Handler":null,"Name":"","Endpoint":""}
Jul 18 00:40:44.895466 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.895446    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 00:40:44.895595 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.895482    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 00:40:44.914449 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.914410    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:44.914615 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.914410    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:44.914615 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:44.914542    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:44.914730 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:44.914419    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:44.914786 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:44.914649    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:44.914786 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:44.914764    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:45.164727 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:45.164621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qqc4k" event={"ID":"6bea8a24-9778-4af6-b6ea-130307c1a405","Type":"ContainerStarted","Data":"3bc3f32d8b62e9a3080d72966096e109a38e3202fa16b3ee71f44cf5eaa88088"}
Jul 18 00:40:45.166491 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:45.166463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b" event={"ID":"3aebfac2-b504-44d8-9f4b-46e0e37e191a","Type":"ContainerStarted","Data":"225f150f26acd4f785884d15cf78eef16b341c8afd79867c2eca747aa7694fdb"}
Jul 18 00:40:45.179160 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:45.179113    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qqc4k" podStartSLOduration=9.822182343 podStartE2EDuration="24.179094873s" totalImagesPullingTime="14.35691253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:45.179039506 +0000 UTC m=+24.706544466" watchObservedRunningTime="2026-07-18 00:40:45.179094873 +0000 UTC m=+24.706599823"
Jul 18 00:40:46.171619 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:46.171589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"f23d5f0c850727878da2df151525e11d3b1dd084de82cd646ba2847d9e5ae31e"}
Jul 18 00:40:46.913085 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:46.912897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:46.913286 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:46.912897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:46.913286 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:46.913175    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:46.913286 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:46.912897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:46.913286 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:46.913240    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:46.913513 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:46.913330    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:47.175906 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:47.175824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b" event={"ID":"3aebfac2-b504-44d8-9f4b-46e0e37e191a","Type":"ContainerStarted","Data":"97ce78bf0ca61f04d9024ff5fa5a7163e5a487dde6d7793028b675c387f05120"}
Jul 18 00:40:47.192673 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:47.192614    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82d2b" podStartSLOduration=4.374686928 podStartE2EDuration="26.192591715s" totalImagesPullingTime="21.817904787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:47.191353944 +0000 UTC m=+26.718858904" watchObservedRunningTime="2026-07-18 00:40:47.192591715 +0000 UTC m=+26.720096676"
Jul 18 00:40:47.685424 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:47.685388    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:47.686051 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:47.686031    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:48.178182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:48.178126    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:48.178700 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:48.178562    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8rx49"
Jul 18 00:40:48.913440 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:48.913407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:48.913618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:48.913530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:48.913618 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:48.913528    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:48.913618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:48.913407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:48.913618 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:48.913605    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:48.913843 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:48.913687    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:49.184717 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:49.184551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" event={"ID":"b5e126a6-ba94-416e-8638-a4a9ecf7b93f","Type":"ContainerStarted","Data":"4e99e412406d2fa618249bb220b116d995718089f52b3fdb5011071b8394ff55"}
Jul 18 00:40:49.212650 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:49.212514    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9" podStartSLOduration=9.176036584 podStartE2EDuration="28.212496022s" totalImagesPullingTime="19.036459438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:49.212128889 +0000 UTC m=+28.739633886" watchObservedRunningTime="2026-07-18 00:40:49.212496022 +0000 UTC m=+28.740000981"
Jul 18 00:40:50.188805 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.188622    2567 generic.go:356] "Generic (PLEG): container finished" podID="177e03e0-5d99-449f-aac8-09b9b7f8f6db" containerID="b7bf483d32ea4f101ca8ebb77a45ed42c40fc431063a2141752e5f86741de091" exitCode=0
Jul 18 00:40:50.188805 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.188696    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerDied","Data":"b7bf483d32ea4f101ca8ebb77a45ed42c40fc431063a2141752e5f86741de091"}
Jul 18 00:40:50.189745 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.189569    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:50.189745 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.189637    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:50.189745 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.189690    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:50.205808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.205785    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:50.209717 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.209699    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:40:50.912729 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.912691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:50.913777 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:50.913745    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:50.913927 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.913829    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:50.913992 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:50.913932    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:50.913992 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:50.913957    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:50.914099 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:50.914007    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:51.358714 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:51.358684    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qb7rp"]
Jul 18 00:40:51.359087 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:51.358813    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:51.359087 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:51.358914    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:51.361737 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:51.361697    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4bl8d"]
Jul 18 00:40:51.361845 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:51.361797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:51.361923 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:51.361904    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:51.374103 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:51.374076    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-45txc"]
Jul 18 00:40:51.374239 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:51.374161    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:51.374239 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:51.374232    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:52.195400 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:52.195367    2567 generic.go:356] "Generic (PLEG): container finished" podID="177e03e0-5d99-449f-aac8-09b9b7f8f6db" containerID="40abc52f916d281d347af6e8ef1f595e61634f0c4aae85a24adf5aeb95f7be98" exitCode=0
Jul 18 00:40:52.195545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:52.195448    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerDied","Data":"40abc52f916d281d347af6e8ef1f595e61634f0c4aae85a24adf5aeb95f7be98"}
Jul 18 00:40:52.917191 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:52.917161    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:52.917191 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:52.917192    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:52.917622 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:52.917192    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:52.917622 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:52.917322    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:52.917622 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:52.917440    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:52.917622 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:52.917508    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:53.200514 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:53.200484    2567 generic.go:356] "Generic (PLEG): container finished" podID="177e03e0-5d99-449f-aac8-09b9b7f8f6db" containerID="628184ef9dcdd42807455cdbfb837f3168432eaee42c962aa87938203ad0fbe7" exitCode=0
Jul 18 00:40:53.200672 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:53.200521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerDied","Data":"628184ef9dcdd42807455cdbfb837f3168432eaee42c962aa87938203ad0fbe7"}
Jul 18 00:40:54.686772 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:54.686732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:54.687347 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:54.686787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:54.687347 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.686907    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:54.687347 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.686916    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:54.687347 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.686980    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:26.68696435 +0000 UTC m=+66.214469290 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 00:40:54.687347 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.686993    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret podName:d0098140-e04e-4d1c-b440-09c25f6825f9 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:26.686987742 +0000 UTC m=+66.214492683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret") pod "global-pull-secret-syncer-4bl8d" (UID: "d0098140-e04e-4d1c-b440-09c25f6825f9") : object "kube-system"/"original-pull-secret" not registered
Jul 18 00:40:54.787861 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:54.787799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:54.788034 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.787974    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 00:40:54.788034 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.787998    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 00:40:54.788034 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.788007    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4pnnh for pod openshift-network-diagnostics/network-check-target-45txc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:54.788162 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.788062    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh podName:68797949-9c2b-4cdd-b18e-b8cb61f8d676 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:26.788046647 +0000 UTC m=+66.315551585 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pnnh" (UniqueName: "kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh") pod "network-check-target-45txc" (UID: "68797949-9c2b-4cdd-b18e-b8cb61f8d676") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 00:40:54.913721 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:54.913686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:54.913894 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:54.913731    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:54.913894 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.913814    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45txc" podUID="68797949-9c2b-4cdd-b18e-b8cb61f8d676"
Jul 18 00:40:54.914232 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.914207    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bl8d" podUID="d0098140-e04e-4d1c-b440-09c25f6825f9"
Jul 18 00:40:54.914307 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:54.914290    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:54.914402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:54.914383    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qb7rp" podUID="984d0621-5433-4613-b1ff-41f803ca0a53"
Jul 18 00:40:55.834022 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.833990    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-55.ec2.internal" event="NodeReady"
Jul 18 00:40:55.834441 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.834162    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 00:40:55.870423 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.870389    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66945fc9c5-q5mdr"]
Jul 18 00:40:55.874770 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.874740    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.876566 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.876524    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-km4hb"]
Jul 18 00:40:55.877498 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.877403    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 18 00:40:55.879076 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.879051    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 18 00:40:55.879281 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.879251    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 18 00:40:55.879484 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.879468    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h4ffv\""
Jul 18 00:40:55.880837 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.880816    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"]
Jul 18 00:40:55.880986 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.880970    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.884134 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.884112    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gj8gh\""
Jul 18 00:40:55.884509 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.884490    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 18 00:40:55.885541 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.884856    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.885541 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.885431    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 18 00:40:55.885541 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.885513    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"]
Jul 18 00:40:55.885804 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.885784    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"
Jul 18 00:40:55.885916 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.885897    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.887194 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.887176    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 18 00:40:55.888773 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.888368    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.888773 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.888651    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.888923 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.888858    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-chfqk\""
Jul 18 00:40:55.889456 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.889430    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"]
Jul 18 00:40:55.889546 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.889524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:55.891427 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.891409    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 18 00:40:55.891683 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.891661    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8xmqw\""
Jul 18 00:40:55.891900 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.891881    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.892051 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.892018    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 18 00:40:55.893172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.893149    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.893324 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.893302    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 18 00:40:55.893952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.893525    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"]
Jul 18 00:40:55.893952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.893663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:55.897944 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.897919    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8t9b9\""
Jul 18 00:40:55.898707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.898686    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 18 00:40:55.900245 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.900224    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.902619 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.902186    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"]
Jul 18 00:40:55.902619 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.902466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:55.904949 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.904183    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.904949 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.904185    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 18 00:40:55.908644 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.908616    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rrr5c"]
Jul 18 00:40:55.908799 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.908746    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:55.909315 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.909294    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.910400 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.910378    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 18 00:40:55.911617 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.911597    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.912194 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"]
Jul 18 00:40:55.912295 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"]
Jul 18 00:40:55.912295 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912234    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66945fc9c5-q5mdr"]
Jul 18 00:40:55.912295 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912251    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-km4hb"]
Jul 18 00:40:55.912412 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912301    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"]
Jul 18 00:40:55.912412 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912328    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"]
Jul 18 00:40:55.912412 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912345    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rrr5c"]
Jul 18 00:40:55.912412 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.912365    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"]
Jul 18 00:40:55.913107 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.913082    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:55.913444 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.913427    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ctsqr\""
Jul 18 00:40:55.913857 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.913840    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.917222 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.917201    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 18 00:40:55.917620 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.917603    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.918484 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.918463    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7df7df96f4-vqk8n"]
Jul 18 00:40:55.918621 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.918606    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:55.919434 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.919414    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 18 00:40:55.920406 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.920387    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fk5c7\""
Jul 18 00:40:55.920633 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.920618    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 18 00:40:55.922148 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.922128    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"]
Jul 18 00:40:55.922328 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.922310    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:55.922772 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.922712    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 18 00:40:55.923723 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.923417    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.926546 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.926521    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 18 00:40:55.927594 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.927574    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6m5bf"]
Jul 18 00:40:55.927705 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.927691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"
Jul 18 00:40:55.930872 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.930851    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:55.935579 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.935559    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gwcs7\""
Jul 18 00:40:55.936478 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936152    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.936478 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936434    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.936626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936490    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 18 00:40:55.936626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936525    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 18 00:40:55.936720 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936702    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 18 00:40:55.936830 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936812    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zwp6h\""
Jul 18 00:40:55.936912 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.936827    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"]
Jul 18 00:40:55.937306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937077    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.937306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937111    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.937306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937128    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bphpb\""
Jul 18 00:40:55.937306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937078    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 00:40:55.937306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937128    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 18 00:40:55.937540 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937381    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.937699 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937678    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rbpzb\""
Jul 18 00:40:55.937776 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937743    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 00:40:55.937935 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937913    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 18 00:40:55.938003 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.937943    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 00:40:55.942505 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.942482    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-txq7s\""
Jul 18 00:40:55.943738 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.943713    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"]
Jul 18 00:40:55.949296 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.949257    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 18 00:40:55.954677 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.954646    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6m5bf"]
Jul 18 00:40:55.968198 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.968172    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7df7df96f4-vqk8n"]
Jul 18 00:40:55.970222 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.970192    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"]
Jul 18 00:40:55.988440 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.988410    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f8p9z"]
Jul 18 00:40:55.992740 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.992692    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:55.995324 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.995001    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 00:40:55.995324 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.995078    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 00:40:55.995324 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.995202    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9hbbz\""
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.996843    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.996878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c69309b0-d488-49df-9bab-4cdcb1c74e2f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.996911    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cd54\" (UniqueName: \"kubernetes.io/projected/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-kube-api-access-9cd54\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.996938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.996963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.996983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-image-registry-private-configuration\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4ndd\" (UniqueName: \"kubernetes.io/projected/86287ebe-2d1b-4ed6-af4d-622cb20af794-kube-api-access-p4ndd\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997078    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4238d52-036a-4948-88cd-3d2f814d369c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.997141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997102    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4238d52-036a-4948-88cd-3d2f814d369c-tmp\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997442    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997471    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-trusted-ca\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99qv8\" (UniqueName: \"kubernetes.io/projected/1d015041-2e4e-4f50-9bd1-ea921680c85d-kube-api-access-99qv8\") pod \"volume-data-source-validator-78cf4c6f6-sgzwt\" (UID: \"1d015041-2e4e-4f50-9bd1-ea921680c85d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997520    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgp9b\" (UniqueName: \"kubernetes.io/projected/c69309b0-d488-49df-9bab-4cdcb1c74e2f-kube-api-access-mgp9b\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4238d52-036a-4948-88cd-3d2f814d369c-service-ca-bundle\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.997687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4238d52-036a-4948-88cd-3d2f814d369c-snapshots\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwn5k\" (UniqueName: \"kubernetes.io/projected/c4238d52-036a-4948-88cd-3d2f814d369c-kube-api-access-rwn5k\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnnz4\" (UniqueName: \"kubernetes.io/projected/4dab6681-22ef-4518-9bea-f894457cd197-kube-api-access-jnnz4\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997803    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c69309b0-d488-49df-9bab-4cdcb1c74e2f-config\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-installation-pull-secrets\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997858    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-bound-sa-token\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-stats-auth\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-certificates\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.997984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4mr\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-kube-api-access-8z4mr\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.998039 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4238d52-036a-4948-88cd-3d2f814d369c-serving-cert\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998106    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/99bcb8a3-01c9-454e-ad27-17bd14fde331-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998136    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4dab6681-22ef-4518-9bea-f894457cd197-trusted-ca\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bf46\" (UniqueName: \"kubernetes.io/projected/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-kube-api-access-9bf46\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998189    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998214    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4dab6681-22ef-4518-9bea-f894457cd197-config\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998295    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clpmd\" (UniqueName: \"kubernetes.io/projected/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-kube-api-access-clpmd\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-default-certificate\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998363    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-ca-trust-extracted\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:55.998507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:55.998398    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4dab6681-22ef-4518-9bea-f894457cd197-serving-cert\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.001498 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.001471    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f8p9z"]
Jul 18 00:40:56.099305 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099197    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.099305 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4238d52-036a-4948-88cd-3d2f814d369c-service-ca-bundle\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4238d52-036a-4948-88cd-3d2f814d369c-snapshots\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwn5k\" (UniqueName: \"kubernetes.io/projected/c4238d52-036a-4948-88cd-3d2f814d369c-kube-api-access-rwn5k\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099375    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnnz4\" (UniqueName: \"kubernetes.io/projected/4dab6681-22ef-4518-9bea-f894457cd197-kube-api-access-jnnz4\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c69309b0-d488-49df-9bab-4cdcb1c74e2f-config\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099416    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glqz9\" (UniqueName: \"kubernetes.io/projected/6ed78294-e7f5-44af-966b-8ba2d4d6821f-kube-api-access-glqz9\") pod \"ingress-canary-6m5bf\" (UID: \"6ed78294-e7f5-44af-966b-8ba2d4d6821f\") " pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-installation-pull-secrets\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.099524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099484    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-bound-sa-token\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-stats-auth\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-certificates\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gns29\" (UniqueName: \"kubernetes.io/projected/c139f8b3-76a6-47d8-95e6-e134ba1ac190-kube-api-access-gns29\") pod \"network-check-source-7bc5b6bf85-cgwgw\" (UID: \"c139f8b3-76a6-47d8-95e6-e134ba1ac190\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099610    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4mr\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-kube-api-access-8z4mr\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4238d52-036a-4948-88cd-3d2f814d369c-serving-cert\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/666e0686-d4c4-4d89-8ff5-e7de233831be-tmp-dir\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099693    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/99bcb8a3-01c9-454e-ad27-17bd14fde331-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4dab6681-22ef-4518-9bea-f894457cd197-trusted-ca\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bf46\" (UniqueName: \"kubernetes.io/projected/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-kube-api-access-9bf46\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4dab6681-22ef-4518-9bea-f894457cd197-config\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clpmd\" (UniqueName: \"kubernetes.io/projected/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-kube-api-access-clpmd\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/666e0686-d4c4-4d89-8ff5-e7de233831be-config-volume\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099847    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-default-certificate\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.099871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-ca-trust-extracted\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4dab6681-22ef-4518-9bea-f894457cd197-serving-cert\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.099967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c69309b0-d488-49df-9bab-4cdcb1c74e2f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cd54\" (UniqueName: \"kubernetes.io/projected/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-kube-api-access-9cd54\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-image-registry-private-configuration\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4ndd\" (UniqueName: \"kubernetes.io/projected/86287ebe-2d1b-4ed6-af4d-622cb20af794-kube-api-access-p4ndd\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6gc5\" (UniqueName: \"kubernetes.io/projected/666e0686-d4c4-4d89-8ff5-e7de233831be-kube-api-access-p6gc5\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100200    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4238d52-036a-4948-88cd-3d2f814d369c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4238d52-036a-4948-88cd-3d2f814d369c-tmp\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:56.100433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ed78294-e7f5-44af-966b-8ba2d4d6821f-cert\") pod \"ingress-canary-6m5bf\" (UID: \"6ed78294-e7f5-44af-966b-8ba2d4d6821f\") " pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.101027 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100346    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-trusted-ca\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.101027 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100365    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99qv8\" (UniqueName: \"kubernetes.io/projected/1d015041-2e4e-4f50-9bd1-ea921680c85d-kube-api-access-99qv8\") pod \"volume-data-source-validator-78cf4c6f6-sgzwt\" (UID: \"1d015041-2e4e-4f50-9bd1-ea921680c85d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"
Jul 18 00:40:56.101027 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.100381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgp9b\" (UniqueName: \"kubernetes.io/projected/c69309b0-d488-49df-9bab-4cdcb1c74e2f-kube-api-access-mgp9b\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.101027 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.100815    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 00:40:56.101027 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.100834    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66945fc9c5-q5mdr: secret "image-registry-tls" not found
Jul 18 00:40:56.101027 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.100903    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls podName:7fb97b25-d83f-4e59-9dfa-5433ca15cbf7 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.600882714 +0000 UTC m=+36.128387655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls") pod "image-registry-66945fc9c5-q5mdr" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7") : secret "image-registry-tls" not found
Jul 18 00:40:56.101779 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.101605    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:56.101779 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.101701    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.601684063 +0000 UTC m=+36.129189000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:56.101970 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.101939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4238d52-036a-4948-88cd-3d2f814d369c-service-ca-bundle\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.102114 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.102074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c69309b0-d488-49df-9bab-4cdcb1c74e2f-config\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.102215 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.102160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c4238d52-036a-4948-88cd-3d2f814d369c-snapshots\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.102215 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.102185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-certificates\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.102344 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.102316    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.602297849 +0000 UTC m=+36.129802806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : configmap references non-existent config key: service-ca.crt
Jul 18 00:40:56.102500 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.102462    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/99bcb8a3-01c9-454e-ad27-17bd14fde331-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:56.102804 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.102695    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-ca-trust-extracted\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.102892 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.102865    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4dab6681-22ef-4518-9bea-f894457cd197-trusted-ca\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.102940 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.102891    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 00:40:56.102985 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.102954    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.60293577 +0000 UTC m=+36.130440710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : secret "router-metrics-certs-default" not found
Jul 18 00:40:56.103226 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.103200    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4dab6681-22ef-4518-9bea-f894457cd197-config\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.103635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.103904    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-trusted-ca\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.104356    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4238d52-036a-4948-88cd-3d2f814d369c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.104416    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.104466    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.604451644 +0000 UTC m=+36.131956583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.104598    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4238d52-036a-4948-88cd-3d2f814d369c-tmp\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.104688    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.104733    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls podName:86287ebe-2d1b-4ed6-af4d-622cb20af794 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.604718213 +0000 UTC m=+36.132223150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zfm7l" (UID: "86287ebe-2d1b-4ed6-af4d-622cb20af794") : secret "samples-operator-tls" not found
Jul 18 00:40:56.105242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.105191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.107458 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.107430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4dab6681-22ef-4518-9bea-f894457cd197-serving-cert\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.107823 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.107795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c4238d52-036a-4948-88cd-3d2f814d369c-serving-cert\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.107927 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.107909    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-stats-auth\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.108253 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.108229    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c69309b0-d488-49df-9bab-4cdcb1c74e2f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.108510 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.108479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-default-certificate\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.108555 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.108533    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-installation-pull-secrets\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.109161 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.109137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.109766 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.109747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-image-registry-private-configuration\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.109915 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.109898    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgp9b\" (UniqueName: \"kubernetes.io/projected/c69309b0-d488-49df-9bab-4cdcb1c74e2f-kube-api-access-mgp9b\") pod \"service-ca-operator-5bcbc94c78-lsstf\" (UID: \"c69309b0-d488-49df-9bab-4cdcb1c74e2f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.113593 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.113568    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bf46\" (UniqueName: \"kubernetes.io/projected/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-kube-api-access-9bf46\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:56.114862 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.114804    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnnz4\" (UniqueName: \"kubernetes.io/projected/4dab6681-22ef-4518-9bea-f894457cd197-kube-api-access-jnnz4\") pod \"console-operator-565b7b569d-rrr5c\" (UID: \"4dab6681-22ef-4518-9bea-f894457cd197\") " pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.114982 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.114955    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-bound-sa-token\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.116235 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.116216    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwn5k\" (UniqueName: \"kubernetes.io/projected/c4238d52-036a-4948-88cd-3d2f814d369c-kube-api-access-rwn5k\") pod \"insights-operator-555d9d7885-km4hb\" (UID: \"c4238d52-036a-4948-88cd-3d2f814d369c\") " pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.116723 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.116703    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99qv8\" (UniqueName: \"kubernetes.io/projected/1d015041-2e4e-4f50-9bd1-ea921680c85d-kube-api-access-99qv8\") pod \"volume-data-source-validator-78cf4c6f6-sgzwt\" (UID: \"1d015041-2e4e-4f50-9bd1-ea921680c85d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"
Jul 18 00:40:56.117609 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.117387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4mr\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-kube-api-access-8z4mr\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.117900 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.117828    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cd54\" (UniqueName: \"kubernetes.io/projected/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-kube-api-access-9cd54\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.120741 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.120714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clpmd\" (UniqueName: \"kubernetes.io/projected/b36a62f2-c75b-4a7c-97f4-e9dd95ee4254-kube-api-access-clpmd\") pod \"kube-storage-version-migrator-operator-7658d9b749-klc2f\" (UID: \"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.120893 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.120871    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4ndd\" (UniqueName: \"kubernetes.io/projected/86287ebe-2d1b-4ed6-af4d-622cb20af794-kube-api-access-p4ndd\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:56.130587 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.130554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:40:56.198110 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.198076    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-km4hb"
Jul 18 00:40:56.201253 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gns29\" (UniqueName: \"kubernetes.io/projected/c139f8b3-76a6-47d8-95e6-e134ba1ac190-kube-api-access-gns29\") pod \"network-check-source-7bc5b6bf85-cgwgw\" (UID: \"c139f8b3-76a6-47d8-95e6-e134ba1ac190\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"
Jul 18 00:40:56.201419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201308    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/666e0686-d4c4-4d89-8ff5-e7de233831be-tmp-dir\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.201419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201345    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/666e0686-d4c4-4d89-8ff5-e7de233831be-config-volume\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.201624 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6gc5\" (UniqueName: \"kubernetes.io/projected/666e0686-d4c4-4d89-8ff5-e7de233831be-kube-api-access-p6gc5\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.201624 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ed78294-e7f5-44af-966b-8ba2d4d6821f-cert\") pod \"ingress-canary-6m5bf\" (UID: \"6ed78294-e7f5-44af-966b-8ba2d4d6821f\") " pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.201736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glqz9\" (UniqueName: \"kubernetes.io/projected/6ed78294-e7f5-44af-966b-8ba2d4d6821f-kube-api-access-glqz9\") pod \"ingress-canary-6m5bf\" (UID: \"6ed78294-e7f5-44af-966b-8ba2d4d6821f\") " pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.201736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201674    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/666e0686-d4c4-4d89-8ff5-e7de233831be-tmp-dir\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.201736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201688    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.201886 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.201844    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 00:40:56.201931 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.201910    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls podName:666e0686-d4c4-4d89-8ff5-e7de233831be nodeName:}" failed. No retries permitted until 2026-07-18 00:40:56.701892455 +0000 UTC m=+36.229397397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls") pod "dns-default-f8p9z" (UID: "666e0686-d4c4-4d89-8ff5-e7de233831be") : secret "dns-default-metrics-tls" not found
Jul 18 00:40:56.201990 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.201970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/666e0686-d4c4-4d89-8ff5-e7de233831be-config-volume\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.204319 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.204297    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ed78294-e7f5-44af-966b-8ba2d4d6821f-cert\") pod \"ingress-canary-6m5bf\" (UID: \"6ed78294-e7f5-44af-966b-8ba2d4d6821f\") " pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.211007 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.210978    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gns29\" (UniqueName: \"kubernetes.io/projected/c139f8b3-76a6-47d8-95e6-e134ba1ac190-kube-api-access-gns29\") pod \"network-check-source-7bc5b6bf85-cgwgw\" (UID: \"c139f8b3-76a6-47d8-95e6-e134ba1ac190\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"
Jul 18 00:40:56.211151 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.211058    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"
Jul 18 00:40:56.211510 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.211491    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glqz9\" (UniqueName: \"kubernetes.io/projected/6ed78294-e7f5-44af-966b-8ba2d4d6821f-kube-api-access-glqz9\") pod \"ingress-canary-6m5bf\" (UID: \"6ed78294-e7f5-44af-966b-8ba2d4d6821f\") " pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.211827 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.211806    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6gc5\" (UniqueName: \"kubernetes.io/projected/666e0686-d4c4-4d89-8ff5-e7de233831be-kube-api-access-p6gc5\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.218789 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.218765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:40:56.220541 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.220522    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"
Jul 18 00:40:56.243854 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.243826    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"
Jul 18 00:40:56.250930 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.250850    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:40:56.279651 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.279611    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"
Jul 18 00:40:56.285707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.285681    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6m5bf"
Jul 18 00:40:56.606097 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.606060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:56.606350 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.606156    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:56.606350 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.606207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.606350 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606255    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 00:40:56.606350 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606297    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66945fc9c5-q5mdr: secret "image-registry-tls" not found
Jul 18 00:40:56.606350 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606310    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606361    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606367    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls podName:7fb97b25-d83f-4e59-9dfa-5433ca15cbf7 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.606342063 +0000 UTC m=+37.133847002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls") pod "image-registry-66945fc9c5-q5mdr" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7") : secret "image-registry-tls" not found
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.606294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606410    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.60639935 +0000 UTC m=+37.133904288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606443    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.606428052 +0000 UTC m=+37.133932991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : secret "router-metrics-certs-default" not found
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606463    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.606452577 +0000 UTC m=+37.133957519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : configmap references non-existent config key: service-ca.crt
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.606506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:56.606630 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.606576    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:56.606988 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606676    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 00:40:56.606988 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606729    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls podName:86287ebe-2d1b-4ed6-af4d-622cb20af794 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.606716986 +0000 UTC m=+37.134221937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zfm7l" (UID: "86287ebe-2d1b-4ed6-af4d-622cb20af794") : secret "samples-operator-tls" not found
Jul 18 00:40:56.606988 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606758    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:40:56.606988 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.606813    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.606799649 +0000 UTC m=+37.134304598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:40:56.707237 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.707202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:56.707405 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.707355    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 00:40:56.707444 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:56.707417    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls podName:666e0686-d4c4-4d89-8ff5-e7de233831be nodeName:}" failed. No retries permitted until 2026-07-18 00:40:57.707400558 +0000 UTC m=+37.234905496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls") pod "dns-default-f8p9z" (UID: "666e0686-d4c4-4d89-8ff5-e7de233831be") : secret "dns-default-metrics-tls" not found
Jul 18 00:40:56.914037 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.913769    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:40:56.914037 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.913801    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:40:56.914037 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.913787    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:40:56.916255 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.916231    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 00:40:56.916391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.916299    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d2hq6\""
Jul 18 00:40:56.916457 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.916448    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-475p8\""
Jul 18 00:40:56.916540 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:56.916519    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 00:40:57.020484 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.020441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:40:57.075577 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.075539    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"]
Jul 18 00:40:57.079178 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.079148    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"]
Jul 18 00:40:57.079336 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.079304    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.081841 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.081815    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 18 00:40:57.081967 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.081913    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 18 00:40:57.081967 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.081929    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 18 00:40:57.082180 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.082164    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 18 00:40:57.082180 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.082172    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hg9lx\""
Jul 18 00:40:57.083372 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.083349    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"]
Jul 18 00:40:57.083512 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.083497    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.086390 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.086370    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 18 00:40:57.088115 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.087701    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"]
Jul 18 00:40:57.088115 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.087710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.090312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.090078    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 18 00:40:57.090312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.090107    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 18 00:40:57.090312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.090187    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 18 00:40:57.090312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.090078    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 18 00:40:57.090812 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.090749    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"]
Jul 18 00:40:57.091769 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.091678    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:40:57.104748 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.102343    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"]
Jul 18 00:40:57.211068 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.210981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z96w\" (UniqueName: \"kubernetes.io/projected/591b1e59-91e4-4815-b005-b9cd42d37153-kube-api-access-9z96w\") pod \"managed-serviceaccount-addon-agent-666b54f76b-ss5pr\" (UID: \"591b1e59-91e4-4815-b005-b9cd42d37153\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.211068 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/591b1e59-91e4-4815-b005-b9cd42d37153-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-666b54f76b-ss5pr\" (UID: \"591b1e59-91e4-4815-b005-b9cd42d37153\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.211259 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211079    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-ca\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.211259 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f243a49-0287-415e-89b8-2201b0a9a6c1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.211366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211283    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgn4f\" (UniqueName: \"kubernetes.io/projected/8f243a49-0287-415e-89b8-2201b0a9a6c1-kube-api-access-bgn4f\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.211366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1171bb7d-8433-429e-a5a0-2d6add217038-tmp\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.211459 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211404    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1171bb7d-8433-429e-a5a0-2d6add217038-klusterlet-config\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.211459 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.211522 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfmvz\" (UniqueName: \"kubernetes.io/projected/1171bb7d-8433-429e-a5a0-2d6add217038-kube-api-access-bfmvz\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.211522 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211474    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-hub\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.211522 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.211490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.312065 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgn4f\" (UniqueName: \"kubernetes.io/projected/8f243a49-0287-415e-89b8-2201b0a9a6c1-kube-api-access-bgn4f\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.312065 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1171bb7d-8433-429e-a5a0-2d6add217038-tmp\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.312320 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1171bb7d-8433-429e-a5a0-2d6add217038-klusterlet-config\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.312400 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.312452 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfmvz\" (UniqueName: \"kubernetes.io/projected/1171bb7d-8433-429e-a5a0-2d6add217038-kube-api-access-bfmvz\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.312452 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-hub\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.312549 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.312549 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312491    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1171bb7d-8433-429e-a5a0-2d6add217038-tmp\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.312549 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z96w\" (UniqueName: \"kubernetes.io/projected/591b1e59-91e4-4815-b005-b9cd42d37153-kube-api-access-9z96w\") pod \"managed-serviceaccount-addon-agent-666b54f76b-ss5pr\" (UID: \"591b1e59-91e4-4815-b005-b9cd42d37153\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.312694 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/591b1e59-91e4-4815-b005-b9cd42d37153-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-666b54f76b-ss5pr\" (UID: \"591b1e59-91e4-4815-b005-b9cd42d37153\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.312694 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-ca\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.312694 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.312679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f243a49-0287-415e-89b8-2201b0a9a6c1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.313787 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.313666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f243a49-0287-415e-89b8-2201b0a9a6c1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.315720 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.315696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-ca\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.315843 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.315823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-hub\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.315900 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.315876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/591b1e59-91e4-4815-b005-b9cd42d37153-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-666b54f76b-ss5pr\" (UID: \"591b1e59-91e4-4815-b005-b9cd42d37153\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.316079 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.316054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.316079 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.316064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8f243a49-0287-415e-89b8-2201b0a9a6c1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.317526 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.317505    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1171bb7d-8433-429e-a5a0-2d6add217038-klusterlet-config\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.320610 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.320586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z96w\" (UniqueName: \"kubernetes.io/projected/591b1e59-91e4-4815-b005-b9cd42d37153-kube-api-access-9z96w\") pod \"managed-serviceaccount-addon-agent-666b54f76b-ss5pr\" (UID: \"591b1e59-91e4-4815-b005-b9cd42d37153\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.321618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.321599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgn4f\" (UniqueName: \"kubernetes.io/projected/8f243a49-0287-415e-89b8-2201b0a9a6c1-kube-api-access-bgn4f\") pod \"cluster-proxy-proxy-agent-6bd58b68d5-jdxr7\" (UID: \"8f243a49-0287-415e-89b8-2201b0a9a6c1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.321715 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.321694    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfmvz\" (UniqueName: \"kubernetes.io/projected/1171bb7d-8433-429e-a5a0-2d6add217038-kube-api-access-bfmvz\") pod \"klusterlet-addon-workmgr-667cd887f8-95x24\" (UID: \"1171bb7d-8433-429e-a5a0-2d6add217038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.402722 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.402678    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"
Jul 18 00:40:57.412585 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.412529    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:40:57.417965 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.417943    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:40:57.615336 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.615299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:57.615527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.615352    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:57.615527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.615422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:57.615527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.615460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:57.615527 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615508    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615542    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615563    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66945fc9c5-q5mdr: secret "image-registry-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615574    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.615556015 +0000 UTC m=+39.143060973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : configmap references non-existent config key: service-ca.crt
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615549    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615608    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls podName:86287ebe-2d1b-4ed6-af4d-622cb20af794 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.615586728 +0000 UTC m=+39.143091679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zfm7l" (UID: "86287ebe-2d1b-4ed6-af4d-622cb20af794") : secret "samples-operator-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615635    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls podName:7fb97b25-d83f-4e59-9dfa-5433ca15cbf7 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.615625521 +0000 UTC m=+39.143130465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls") pod "image-registry-66945fc9c5-q5mdr" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7") : secret "image-registry-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615652    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.615644068 +0000 UTC m=+39.143149005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.615695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:57.615729 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.615729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:57.616170 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615812    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:40:57.616170 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615850    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.615839447 +0000 UTC m=+39.143344400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:40:57.616170 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615900    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 00:40:57.616170 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.615926    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.615918012 +0000 UTC m=+39.143422954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : secret "router-metrics-certs-default" not found
Jul 18 00:40:57.717182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.717150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:57.717378 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.717356    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 00:40:57.717455 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:57.717442    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls podName:666e0686-d4c4-4d89-8ff5-e7de233831be nodeName:}" failed. No retries permitted until 2026-07-18 00:40:59.717422058 +0000 UTC m=+39.244926997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls") pod "dns-default-f8p9z" (UID: "666e0686-d4c4-4d89-8ff5-e7de233831be") : secret "dns-default-metrics-tls" not found
Jul 18 00:40:57.970157 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:57.969830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:40:58.050042 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.048915    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf"]
Jul 18 00:40:58.061682 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.061581    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt"]
Jul 18 00:40:58.063738 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.062731    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr"]
Jul 18 00:40:58.069133 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.068770    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6m5bf"]
Jul 18 00:40:58.085359 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.085317    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d015041_2e4e_4f50_9bd1_ea921680c85d.slice/crio-b3bda3f7e8b2cab8773f2dbf73303deea2071381f746115a8970eab2c3c102e7 WatchSource:0}: Error finding container b3bda3f7e8b2cab8773f2dbf73303deea2071381f746115a8970eab2c3c102e7: Status 404 returned error can't find the container with id b3bda3f7e8b2cab8773f2dbf73303deea2071381f746115a8970eab2c3c102e7
Jul 18 00:40:58.089677 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.089623    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod591b1e59_91e4_4815_b005_b9cd42d37153.slice/crio-03562eb703154a48b90feae968edca368d1da03ee250703a5f905ec35dcf7aea WatchSource:0}: Error finding container 03562eb703154a48b90feae968edca368d1da03ee250703a5f905ec35dcf7aea: Status 404 returned error can't find the container with id 03562eb703154a48b90feae968edca368d1da03ee250703a5f905ec35dcf7aea
Jul 18 00:40:58.092471 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.091618    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed78294_e7f5_44af_966b_8ba2d4d6821f.slice/crio-5bfb261742b4a805bcb63b1328d1b74806c9cb67f317070e8ba747c3f7de1f7f WatchSource:0}: Error finding container 5bfb261742b4a805bcb63b1328d1b74806c9cb67f317070e8ba747c3f7de1f7f: Status 404 returned error can't find the container with id 5bfb261742b4a805bcb63b1328d1b74806c9cb67f317070e8ba747c3f7de1f7f
Jul 18 00:40:58.093258 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.093190    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"]
Jul 18 00:40:58.098005 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.097897    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-km4hb"]
Jul 18 00:40:58.098005 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.097925    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1171bb7d_8433_429e_a5a0_2d6add217038.slice/crio-cbc75c491f6754ba175529643209db395cea9b20145b3dc2589062ef88223361 WatchSource:0}: Error finding container cbc75c491f6754ba175529643209db395cea9b20145b3dc2589062ef88223361: Status 404 returned error can't find the container with id cbc75c491f6754ba175529643209db395cea9b20145b3dc2589062ef88223361
Jul 18 00:40:58.103461 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.103109    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw"]
Jul 18 00:40:58.103705 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.103646    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4238d52_036a_4948_88cd_3d2f814d369c.slice/crio-96921335c8f79d0b902d93a98c4a3e3b854a49e9f7719b98f01b8c46125c38b4 WatchSource:0}: Error finding container 96921335c8f79d0b902d93a98c4a3e3b854a49e9f7719b98f01b8c46125c38b4: Status 404 returned error can't find the container with id 96921335c8f79d0b902d93a98c4a3e3b854a49e9f7719b98f01b8c46125c38b4
Jul 18 00:40:58.109958 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.109818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc139f8b3_76a6_47d8_95e6_e134ba1ac190.slice/crio-97bf8bdb3a79aca490931c95637a4e43d88e67548cba832994bd4302fe56d536 WatchSource:0}: Error finding container 97bf8bdb3a79aca490931c95637a4e43d88e67548cba832994bd4302fe56d536: Status 404 returned error can't find the container with id 97bf8bdb3a79aca490931c95637a4e43d88e67548cba832994bd4302fe56d536
Jul 18 00:40:58.124642 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.124568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"]
Jul 18 00:40:58.128120 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.128089    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f"]
Jul 18 00:40:58.132553 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.132525    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rrr5c"]
Jul 18 00:40:58.132633 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.132611    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36a62f2_c75b_4a7c_97f4_e9dd95ee4254.slice/crio-1cbe9a312c17e699bc7fdf9d4ef3b504a38a1171e8eaca7261e0e519c686b501 WatchSource:0}: Error finding container 1cbe9a312c17e699bc7fdf9d4ef3b504a38a1171e8eaca7261e0e519c686b501: Status 404 returned error can't find the container with id 1cbe9a312c17e699bc7fdf9d4ef3b504a38a1171e8eaca7261e0e519c686b501
Jul 18 00:40:58.135846 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:40:58.135826    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dab6681_22ef_4518_9bea_f894457cd197.slice/crio-4f3c23c07be4ef632bfa1ef3ebd07296ded84bbe32b58e42bb2411283c861894 WatchSource:0}: Error finding container 4f3c23c07be4ef632bfa1ef3ebd07296ded84bbe32b58e42bb2411283c861894: Status 404 returned error can't find the container with id 4f3c23c07be4ef632bfa1ef3ebd07296ded84bbe32b58e42bb2411283c861894
Jul 18 00:40:58.222118 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.222077    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf" event={"ID":"c69309b0-d488-49df-9bab-4cdcb1c74e2f","Type":"ContainerStarted","Data":"7bc9a95c917c49b3a4727f4f4c858bdc2983947aca078ad6ad0db11a3e00b5b2"}
Jul 18 00:40:58.223569 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.223543    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" event={"ID":"4dab6681-22ef-4518-9bea-f894457cd197","Type":"ContainerStarted","Data":"4f3c23c07be4ef632bfa1ef3ebd07296ded84bbe32b58e42bb2411283c861894"}
Jul 18 00:40:58.224692 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.224658    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-km4hb" event={"ID":"c4238d52-036a-4948-88cd-3d2f814d369c","Type":"ContainerStarted","Data":"96921335c8f79d0b902d93a98c4a3e3b854a49e9f7719b98f01b8c46125c38b4"}
Jul 18 00:40:58.227852 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.227833    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerStarted","Data":"30a005104ef0f7d691505974a3495e9791a918ce4c1e23e8fb3bd2214e950608"}
Jul 18 00:40:58.228980 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.228962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24" event={"ID":"1171bb7d-8433-429e-a5a0-2d6add217038","Type":"ContainerStarted","Data":"cbc75c491f6754ba175529643209db395cea9b20145b3dc2589062ef88223361"}
Jul 18 00:40:58.230167 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.230141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6m5bf" event={"ID":"6ed78294-e7f5-44af-966b-8ba2d4d6821f","Type":"ContainerStarted","Data":"5bfb261742b4a805bcb63b1328d1b74806c9cb67f317070e8ba747c3f7de1f7f"}
Jul 18 00:40:58.231238 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.231214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f" event={"ID":"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254","Type":"ContainerStarted","Data":"1cbe9a312c17e699bc7fdf9d4ef3b504a38a1171e8eaca7261e0e519c686b501"}
Jul 18 00:40:58.238171 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.238143    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" event={"ID":"8f243a49-0287-415e-89b8-2201b0a9a6c1","Type":"ContainerStarted","Data":"c7122894190d676be5df1d2beeef83e65e581196f3b647c548503849bf314177"}
Jul 18 00:40:58.239224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.239206    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr" event={"ID":"591b1e59-91e4-4815-b005-b9cd42d37153","Type":"ContainerStarted","Data":"03562eb703154a48b90feae968edca368d1da03ee250703a5f905ec35dcf7aea"}
Jul 18 00:40:58.240296 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.240256    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw" event={"ID":"c139f8b3-76a6-47d8-95e6-e134ba1ac190","Type":"ContainerStarted","Data":"97bf8bdb3a79aca490931c95637a4e43d88e67548cba832994bd4302fe56d536"}
Jul 18 00:40:58.241256 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:58.241237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt" event={"ID":"1d015041-2e4e-4f50-9bd1-ea921680c85d","Type":"ContainerStarted","Data":"b3bda3f7e8b2cab8773f2dbf73303deea2071381f746115a8970eab2c3c102e7"}
Jul 18 00:40:59.278106 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.277844    2567 generic.go:356] "Generic (PLEG): container finished" podID="177e03e0-5d99-449f-aac8-09b9b7f8f6db" containerID="30a005104ef0f7d691505974a3495e9791a918ce4c1e23e8fb3bd2214e950608" exitCode=0
Jul 18 00:40:59.278106 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.278095    2567 generic.go:356] "Generic (PLEG): container finished" podID="177e03e0-5d99-449f-aac8-09b9b7f8f6db" containerID="d7faeb64de66080835488aeab9853b8cbe16a9c36e69451a181230bd1c25ec93" exitCode=0
Jul 18 00:40:59.278664 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.277979    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerDied","Data":"30a005104ef0f7d691505974a3495e9791a918ce4c1e23e8fb3bd2214e950608"}
Jul 18 00:40:59.278664 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.278192    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerDied","Data":"d7faeb64de66080835488aeab9853b8cbe16a9c36e69451a181230bd1c25ec93"}
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.643786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.643852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.643936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.643983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.644036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.644077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:40:59.644369 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.644241    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.644940    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.644970    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.644985    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66945fc9c5-q5mdr: secret "image-registry-tls" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645013    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls podName:86287ebe-2d1b-4ed6-af4d-622cb20af794 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.644992321 +0000 UTC m=+43.172497279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zfm7l" (UID: "86287ebe-2d1b-4ed6-af4d-622cb20af794") : secret "samples-operator-tls" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645031    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.645022242 +0000 UTC m=+43.172527180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : configmap references non-existent config key: service-ca.crt
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645043    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls podName:7fb97b25-d83f-4e59-9dfa-5433ca15cbf7 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.645036425 +0000 UTC m=+43.172541363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls") pod "image-registry-66945fc9c5-q5mdr" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7") : secret "image-registry-tls" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645084    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645103    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:59.645146 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645142    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.645124741 +0000 UTC m=+43.172629690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : secret "router-metrics-certs-default" not found
Jul 18 00:40:59.646346 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.645163    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.645153561 +0000 UTC m=+43.172658498 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:40:59.646407 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.646361    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.646310497 +0000 UTC m=+43.173815441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:40:59.744763 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:40:59.744726    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:40:59.744943 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.744903    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 00:40:59.745001 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:40:59.744970    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls podName:666e0686-d4c4-4d89-8ff5-e7de233831be nodeName:}" failed. No retries permitted until 2026-07-18 00:41:03.744950499 +0000 UTC m=+43.272455460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls") pod "dns-default-f8p9z" (UID: "666e0686-d4c4-4d89-8ff5-e7de233831be") : secret "dns-default-metrics-tls" not found
Jul 18 00:41:00.331943 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:00.331859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-smk9b" event={"ID":"177e03e0-5d99-449f-aac8-09b9b7f8f6db","Type":"ContainerStarted","Data":"c41b3e62314b37106fc2a345c66d44a085bdcd7db0ead4c9643f2055887009b1"}
Jul 18 00:41:00.358575 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:00.358515    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-smk9b" podStartSLOduration=8.742302623 podStartE2EDuration="39.358490577s" totalImagesPullingTime="30.616187954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:00.35618553 +0000 UTC m=+39.883690490" watchObservedRunningTime="2026-07-18 00:41:00.358490577 +0000 UTC m=+39.885995543"
Jul 18 00:41:00.483170 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:00.483134    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:00.547071 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:00.546977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:00.975633 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:00.975520    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:03.530773 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.530728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:03.590624 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.590591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:03.691796 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.691767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:03.693891 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.693870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:41:03.693951 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.693907    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:03.693982 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.693955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:41:03.694015 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.693985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:03.694055 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694014    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 00:41:03.694091 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694066    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls podName:86287ebe-2d1b-4ed6-af4d-622cb20af794 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.694052631 +0000 UTC m=+51.221557589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zfm7l" (UID: "86287ebe-2d1b-4ed6-af4d-622cb20af794") : secret "samples-operator-tls" not found
Jul 18 00:41:03.694091 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694082    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:41:03.694166 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.694016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:03.694166 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694091    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.694080443 +0000 UTC m=+51.221585382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : configmap references non-existent config key: service-ca.crt
Jul 18 00:41:03.694166 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694085    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 00:41:03.694166 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694140    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 00:41:03.694374 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694144    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66945fc9c5-q5mdr: secret "image-registry-tls" not found
Jul 18 00:41:03.694374 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694166    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.694151709 +0000 UTC m=+51.221656650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:41:03.694374 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694293    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.694254735 +0000 UTC m=+51.221759678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : secret "router-metrics-certs-default" not found
Jul 18 00:41:03.694374 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694330    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls podName:7fb97b25-d83f-4e59-9dfa-5433ca15cbf7 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.694318519 +0000 UTC m=+51.221823461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls") pod "image-registry-66945fc9c5-q5mdr" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7") : secret "image-registry-tls" not found
Jul 18 00:41:03.694568 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.694391    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:41:03.694568 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694521    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:41:03.694568 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.694568    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.694556273 +0000 UTC m=+51.222061215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:41:03.795501 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:03.795418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:03.795634 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.795575    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 00:41:03.795682 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:03.795643    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls podName:666e0686-d4c4-4d89-8ff5-e7de233831be nodeName:}" failed. No retries permitted until 2026-07-18 00:41:11.795628453 +0000 UTC m=+51.323133394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls") pod "dns-default-f8p9z" (UID: "666e0686-d4c4-4d89-8ff5-e7de233831be") : secret "dns-default-metrics-tls" not found
Jul 18 00:41:05.543337 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:05.543303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:05.605695 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:05.605657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:06.698440 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:06.698371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:06.799082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:06.799032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:07.473131 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:07.473100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:10.111652 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:10.111612    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:10.211808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:10.211776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:11.776010 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.775972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.776055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.776103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776151    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776167    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66945fc9c5-q5mdr: secret "image-registry-tls" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776210    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776215    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.776152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776229    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls podName:7fb97b25-d83f-4e59-9dfa-5433ca15cbf7 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.776209347 +0000 UTC m=+67.303714302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls") pod "image-registry-66945fc9c5-q5mdr" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7") : secret "image-registry-tls" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776320    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.776306511 +0000 UTC m=+67.303811452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : secret "router-metrics-certs-default" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776338    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.776328597 +0000 UTC m=+67.303833542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776356    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle podName:7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.776347823 +0000 UTC m=+67.303852763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle") pod "router-default-7df7df96f4-vqk8n" (UID: "7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608") : configmap references non-existent config key: service-ca.crt
Jul 18 00:41:11.776402 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.776402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:41:11.776826 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.776466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:41:11.776826 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776509    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:41:11.776826 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776548    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.776534452 +0000 UTC m=+67.304039389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:41:11.776826 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776561    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 00:41:11.776826 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.776598    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls podName:86287ebe-2d1b-4ed6-af4d-622cb20af794 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.776585689 +0000 UTC m=+67.304090629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-zfm7l" (UID: "86287ebe-2d1b-4ed6-af4d-622cb20af794") : secret "samples-operator-tls" not found
Jul 18 00:41:11.877532 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:11.877496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:11.877690 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.877667    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 00:41:11.877748 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:11.877739    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls podName:666e0686-d4c4-4d89-8ff5-e7de233831be nodeName:}" failed. No retries permitted until 2026-07-18 00:41:27.877723669 +0000 UTC m=+67.405228610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls") pod "dns-default-f8p9z" (UID: "666e0686-d4c4-4d89-8ff5-e7de233831be") : secret "dns-default-metrics-tls" not found
Jul 18 00:41:12.383532 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.383389    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6m5bf" event={"ID":"6ed78294-e7f5-44af-966b-8ba2d4d6821f","Type":"ContainerStarted","Data":"d6206e3c4fc8cfdfe6dd323f08fd60b21f7136d898c545f77765ec3a56f49d6f"}
Jul 18 00:41:12.385232 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.385137    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f" event={"ID":"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254","Type":"ContainerStarted","Data":"93b8488842f7e85d24603fd081f73f780768426754b4c5f43aabb396834ff8ce"}
Jul 18 00:41:12.395416 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.395328    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt" event={"ID":"1d015041-2e4e-4f50-9bd1-ea921680c85d","Type":"ContainerStarted","Data":"f7a24937eaff6572867b487b5091a7d4bb840e261ce9605b483f784fd016b7d2"}
Jul 18 00:41:12.398349 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.397446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf" event={"ID":"c69309b0-d488-49df-9bab-4cdcb1c74e2f","Type":"ContainerStarted","Data":"a1828a1a878bb2c3c222903e61b8349f7f3c59eba8f4ac4daf9e89a3b7385a3e"}
Jul 18 00:41:12.405448 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.404927    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f" podStartSLOduration=34.446927254 podStartE2EDuration="48.404907466s" totalImagesPullingTime="13.957980212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:12.403628367 +0000 UTC m=+51.931133328" watchObservedRunningTime="2026-07-18 00:41:12.404907466 +0000 UTC m=+51.932412428"
Jul 18 00:41:12.432221 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.430950    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6m5bf" podStartSLOduration=3.4376569 podStartE2EDuration="17.430929305s" totalImagesPullingTime="13.993272405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:12.429684831 +0000 UTC m=+51.957189772" watchObservedRunningTime="2026-07-18 00:41:12.430929305 +0000 UTC m=+51.958434266"
Jul 18 00:41:12.454380 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.454332    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sgzwt" podStartSLOduration=41.880769044 podStartE2EDuration="48.45431338s" totalImagesPullingTime="6.573544336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:12.449888964 +0000 UTC m=+51.977393925" watchObservedRunningTime="2026-07-18 00:41:12.45431338 +0000 UTC m=+51.981818341"
Jul 18 00:41:12.476287 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.469670    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf" podStartSLOduration=34.46425228 podStartE2EDuration="48.469648783s" totalImagesPullingTime="14.005396503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:12.467759313 +0000 UTC m=+51.995264275" watchObservedRunningTime="2026-07-18 00:41:12.469648783 +0000 UTC m=+51.997153724"
Jul 18 00:41:12.521548 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.521447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:12.619089 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.619060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:12.719808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:12.719730    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:13.405527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.405487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw" event={"ID":"c139f8b3-76a6-47d8-95e6-e134ba1ac190","Type":"ContainerStarted","Data":"42ccb6ef3d6cb8cd915e5bdf921ffafa8f6054c9986e2ef982115fc3fd9099ef"}
Jul 18 00:41:13.407603 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.407571    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/0.log"
Jul 18 00:41:13.407736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.407615    2567 generic.go:356] "Generic (PLEG): container finished" podID="4dab6681-22ef-4518-9bea-f894457cd197" containerID="74c7f239b6fb846bfcb210d298bd5b4d3934f121853a147ea28d55dedb078fa9" exitCode=255
Jul 18 00:41:13.407736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.407690    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" event={"ID":"4dab6681-22ef-4518-9bea-f894457cd197","Type":"ContainerDied","Data":"74c7f239b6fb846bfcb210d298bd5b4d3934f121853a147ea28d55dedb078fa9"}
Jul 18 00:41:13.407962 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.407931    2567 scope.go:117] "RemoveContainer" containerID="74c7f239b6fb846bfcb210d298bd5b4d3934f121853a147ea28d55dedb078fa9"
Jul 18 00:41:13.410703 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.410676    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-km4hb" event={"ID":"c4238d52-036a-4948-88cd-3d2f814d369c","Type":"ContainerStarted","Data":"fb39c8eedcd2a7a79ff3261d0cb747adc189a332b91b9854e9fda11d942e018e"}
Jul 18 00:41:13.413336 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.413313    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24" event={"ID":"1171bb7d-8433-429e-a5a0-2d6add217038","Type":"ContainerStarted","Data":"b93a63eb398c57fc78dc31e19cf24f7bc943edff3384422741dc760a38224cb5"}
Jul 18 00:41:13.413753 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.413739    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:41:13.415792 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.415773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24"
Jul 18 00:41:13.415943 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.415926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" event={"ID":"8f243a49-0287-415e-89b8-2201b0a9a6c1","Type":"ContainerStarted","Data":"a2e9ceb707eac642b1aac8ac303f95adc79267182a8f1b9d95fbc6cb1a35d362"}
Jul 18 00:41:13.415987 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.415954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" event={"ID":"8f243a49-0287-415e-89b8-2201b0a9a6c1","Type":"ContainerStarted","Data":"06e9b35a1687f5a8eafffe80bc893a6d953955b5a86a419d76a08124077701ba"}
Jul 18 00:41:13.415987 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.415970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" event={"ID":"8f243a49-0287-415e-89b8-2201b0a9a6c1","Type":"ContainerStarted","Data":"ee9e3e0192a4f9f0c7c9925078bf36df3f55c2faf7eb8b35e9a35f18e833dba1"}
Jul 18 00:41:13.417973 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.417932    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr" event={"ID":"591b1e59-91e4-4815-b005-b9cd42d37153","Type":"ContainerStarted","Data":"b16f10c0bedf99ec333ec89efc2f7a8defaebd5b5aec25bc7e6183c165464b30"}
Jul 18 00:41:13.429221 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.429154    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cgwgw" podStartSLOduration=35.373097392 podStartE2EDuration="49.429133941s" totalImagesPullingTime="14.056036549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:13.426914343 +0000 UTC m=+52.954419305" watchObservedRunningTime="2026-07-18 00:41:13.429133941 +0000 UTC m=+52.956638902"
Jul 18 00:41:13.459551 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.459479    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-km4hb" podStartSLOduration=36.000781835 podStartE2EDuration="49.459458076s" totalImagesPullingTime="13.458676241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:13.458007579 +0000 UTC m=+52.985512563" watchObservedRunningTime="2026-07-18 00:41:13.459458076 +0000 UTC m=+52.986963036"
Jul 18 00:41:13.460454 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.460414    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-666b54f76b-ss5pr" podStartSLOduration=2.378983825 podStartE2EDuration="16.460385077s" totalImagesPullingTime="14.081401252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:13.441104484 +0000 UTC m=+52.968609437" watchObservedRunningTime="2026-07-18 00:41:13.460385077 +0000 UTC m=+52.987890038"
Jul 18 00:41:13.480366 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.480297    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-667cd887f8-95x24" podStartSLOduration=2.412988373 podStartE2EDuration="16.480258452s" totalImagesPullingTime="14.067270079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:13.47902883 +0000 UTC m=+53.006533812" watchObservedRunningTime="2026-07-18 00:41:13.480258452 +0000 UTC m=+53.007763414"
Jul 18 00:41:13.515812 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:13.515758    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" podStartSLOduration=2.479669788 podStartE2EDuration="16.51573389s" totalImagesPullingTime="14.036064102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:13.513818594 +0000 UTC m=+53.041323556" watchObservedRunningTime="2026-07-18 00:41:13.51573389 +0000 UTC m=+53.043238851"
Jul 18 00:41:14.033773 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.033728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.131438 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.131400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.231311 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.231261    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.422818 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.422786    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/1.log"
Jul 18 00:41:14.423258 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.423242    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/0.log"
Jul 18 00:41:14.423346 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.423311    2567 generic.go:356] "Generic (PLEG): container finished" podID="4dab6681-22ef-4518-9bea-f894457cd197" containerID="6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c" exitCode=255
Jul 18 00:41:14.423433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.423409    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" event={"ID":"4dab6681-22ef-4518-9bea-f894457cd197","Type":"ContainerDied","Data":"6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c"}
Jul 18 00:41:14.423499 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.423470    2567 scope.go:117] "RemoveContainer" containerID="74c7f239b6fb846bfcb210d298bd5b4d3934f121853a147ea28d55dedb078fa9"
Jul 18 00:41:14.423691 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.423674    2567 scope.go:117] "RemoveContainer" containerID="6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c"
Jul 18 00:41:14.423888 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:14.423865    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rrr5c_openshift-console-operator(4dab6681-22ef-4518-9bea-f894457cd197)\"" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podUID="4dab6681-22ef-4518-9bea-f894457cd197"
Jul 18 00:41:14.562418 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.562387    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7g9c6"]
Jul 18 00:41:14.594154 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.594111    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7g9c6"]
Jul 18 00:41:14.594344 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.594255    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.596479 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.596448    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 00:41:14.596618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.596448    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 00:41:14.596618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.596521    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w2tbv\""
Jul 18 00:41:14.615055 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.615027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.666314 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.666263    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.713498 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.713394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.713498 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.713473    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e302077-35e3-437a-88ea-a5102042948b-crio-socket\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.713731 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.713538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e302077-35e3-437a-88ea-a5102042948b-data-volume\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.713731 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.713591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcqb2\" (UniqueName: \"kubernetes.io/projected/0e302077-35e3-437a-88ea-a5102042948b-kube-api-access-qcqb2\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.713816 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.713737    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.766171 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.766135    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.814382 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814342    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.814576 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.814576 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814433    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e302077-35e3-437a-88ea-a5102042948b-crio-socket\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.814576 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:14.814465    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 00:41:14.814576 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e302077-35e3-437a-88ea-a5102042948b-data-volume\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.814576 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814504    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcqb2\" (UniqueName: \"kubernetes.io/projected/0e302077-35e3-437a-88ea-a5102042948b-kube-api-access-qcqb2\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.814576 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:14.814525    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:15.314508508 +0000 UTC m=+54.842013446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : configmap "kube-rbac-proxy" not found
Jul 18 00:41:14.814845 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814581    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e302077-35e3-437a-88ea-a5102042948b-crio-socket\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.814845 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:14.814621    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:14.814845 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:14.814677    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:15.314658856 +0000 UTC m=+54.842163797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:14.814961 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.814852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e302077-35e3-437a-88ea-a5102042948b-data-volume\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:14.817506 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.817484    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:14.826996 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:14.826961    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcqb2\" (UniqueName: \"kubernetes.io/projected/0e302077-35e3-437a-88ea-a5102042948b-kube-api-access-qcqb2\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:15.160872 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.160840    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:15.319873 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.319835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:15.320068 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.319987    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:15.320068 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:15.320010    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:15.320068 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:15.320059    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 00:41:15.320207 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:15.320090    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:16.320069036 +0000 UTC m=+55.847574015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:15.320207 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:15.320114    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:16.320101719 +0000 UTC m=+55.847606657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : configmap "kube-rbac-proxy" not found
Jul 18 00:41:15.428597 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.428522    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/1.log"
Jul 18 00:41:15.429023 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.428922    2567 scope.go:117] "RemoveContainer" containerID="6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c"
Jul 18 00:41:15.429104 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:15.429080    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rrr5c_openshift-console-operator(4dab6681-22ef-4518-9bea-f894457cd197)\"" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podUID="4dab6681-22ef-4518-9bea-f894457cd197"
Jul 18 00:41:15.440032 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.440000    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:15.542218 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:15.542188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:16.193940 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.193912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:16.251495 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.251452    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:41:16.251672 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.251512    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:41:16.293946 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.293913    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:16.329198 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.329164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:16.329316 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.329213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:16.329357 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:16.329317    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 00:41:16.329390 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:16.329360    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:16.329390 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:16.329389    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:18.329373681 +0000 UTC m=+57.856878622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : configmap "kube-rbac-proxy" not found
Jul 18 00:41:16.329461 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:16.329405    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:18.329397742 +0000 UTC m=+57.856902680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:16.432281 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:16.432242    2567 scope.go:117] "RemoveContainer" containerID="6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c"
Jul 18 00:41:16.432626 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:16.432424    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rrr5c_openshift-console-operator(4dab6681-22ef-4518-9bea-f894457cd197)\"" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podUID="4dab6681-22ef-4518-9bea-f894457cd197"
Jul 18 00:41:17.072978 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:17.072948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:17.136771 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:17.136729    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:17.165469 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:17.165449    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdrwg_a8aa0034-92b0-46ce-9347-71f922623496/dns-node-resolver/0.log"
Jul 18 00:41:17.765492 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:17.765466    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fwvdq_0ed34f12-7e48-43c6-b50d-bbe40913298b/node-ca/0.log"
Jul 18 00:41:18.033189 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:18.033101    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:18.349511 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:18.349485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:18.349696 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:18.349531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:18.349696 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:18.349620    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 00:41:18.349696 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:18.349652    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:18.349816 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:18.349713    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:22.34967864 +0000 UTC m=+61.877183578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:18.349816 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:18.349737    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:22.349726336 +0000 UTC m=+61.877231274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : configmap "kube-rbac-proxy" not found
Jul 18 00:41:18.562006 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:18.561975    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6m5bf_6ed78294-e7f5-44af-966b-8ba2d4d6821f/serve-healthcheck-canary/0.log"
Jul 18 00:41:19.363108 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:19.363073    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-klc2f_b36a62f2-c75b-4a7c-97f4-e9dd95ee4254/kube-storage-version-migrator-operator/0.log"
Jul 18 00:41:20.460356 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:20.460321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:20.530419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:20.530384    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.033728 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.033693    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.133836 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.133796    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.235971 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.235939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.421999 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.421967    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.543610 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.543569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.701781 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.701689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:21.986160 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:21.986081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:22.213610 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:22.213583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9pr9"
Jul 18 00:41:22.387834 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:22.387800    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:22.388028 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:22.387862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:22.388028 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:22.387982    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 00:41:22.388028 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:22.388004    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:22.388180 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:22.388084    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:30.388062228 +0000 UTC m=+69.915567182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:22.388180 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:22.388103    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:30.388093831 +0000 UTC m=+69.915598772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : configmap "kube-rbac-proxy" not found
Jul 18 00:41:22.580656 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:22.580627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:23.179867 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:23.179830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:23.377124 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:23.377093    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:23.774863 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:23.774836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:23.979318 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:23.979256    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:24.371843 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:24.371810    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:24.888837 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:24.888798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:24.990621 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:24.990586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:25.093197 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:25.093158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:25.192605 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:25.192524    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:25.782047 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:25.782013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:25.832026 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:25.831998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:26.182968 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.182939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:26.283208 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.283172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:26.342429 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.342393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:26.571776 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.571745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:26.730007 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.729950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:41:26.730186 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.730082    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:41:26.732690 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.732672    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 00:41:26.732740 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.732722    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 00:41:26.740727 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:26.740692    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 00:41:26.740874 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:26.740782    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs podName:984d0621-5433-4613-b1ff-41f803ca0a53 nodeName:}" failed. No retries permitted until 2026-07-18 00:42:30.74075952 +0000 UTC m=+130.268264479 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs") pod "network-metrics-daemon-qb7rp" (UID: "984d0621-5433-4613-b1ff-41f803ca0a53") : secret "metrics-daemon-secret" not found
Jul 18 00:41:26.743052 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.743020    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0098140-e04e-4d1c-b440-09c25f6825f9-original-pull-secret\") pod \"global-pull-secret-syncer-4bl8d\" (UID: \"d0098140-e04e-4d1c-b440-09c25f6825f9\") " pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:41:26.776755 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.776721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:26.831087 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.830998    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:41:26.833677 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.833649    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pnnh\" (UniqueName: \"kubernetes.io/projected/68797949-9c2b-4cdd-b18e-b8cb61f8d676-kube-api-access-4pnnh\") pod \"network-check-target-45txc\" (UID: \"68797949-9c2b-4cdd-b18e-b8cb61f8d676\") " pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:41:26.928405 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.928376    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-475p8\""
Jul 18 00:41:26.936666 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.936638    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:41:26.939946 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:26.939922    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bl8d"
Jul 18 00:41:27.076386 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.076361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-45txc"]
Jul 18 00:41:27.078968 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:41:27.078939    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68797949_9c2b_4cdd_b18e_b8cb61f8d676.slice/crio-171d3a2d0424b7c8da7eb5898ee977ab308b2462e950fe7ab0f8d839e9aff1d2 WatchSource:0}: Error finding container 171d3a2d0424b7c8da7eb5898ee977ab308b2462e950fe7ab0f8d839e9aff1d2: Status 404 returned error can't find the container with id 171d3a2d0424b7c8da7eb5898ee977ab308b2462e950fe7ab0f8d839e9aff1d2
Jul 18 00:41:27.095551 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.095527    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4bl8d"]
Jul 18 00:41:27.099222 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:41:27.099187    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0098140_e04e_4d1c_b440_09c25f6825f9.slice/crio-be15e2fc10c6593e0185c4979330c6e3b26e27f336ffbef466788865e505bfd2 WatchSource:0}: Error finding container be15e2fc10c6593e0185c4979330c6e3b26e27f336ffbef466788865e505bfd2: Status 404 returned error can't find the container with id be15e2fc10c6593e0185c4979330c6e3b26e27f336ffbef466788865e505bfd2
Jul 18 00:41:27.185941 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.185908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:27.378633 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.378554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:27.469221 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.469183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4bl8d" event={"ID":"d0098140-e04e-4d1c-b440-09c25f6825f9","Type":"ContainerStarted","Data":"be15e2fc10c6593e0185c4979330c6e3b26e27f336ffbef466788865e505bfd2"}
Jul 18 00:41:27.470638 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.470613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-45txc" event={"ID":"68797949-9c2b-4cdd-b18e-b8cb61f8d676","Type":"ContainerStarted","Data":"47f1ade5aac4974deb6f45bfeb57f780443962b239edaf3767a324fbcd47b552"}
Jul 18 00:41:27.470768 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.470653    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-45txc" event={"ID":"68797949-9c2b-4cdd-b18e-b8cb61f8d676","Type":"ContainerStarted","Data":"171d3a2d0424b7c8da7eb5898ee977ab308b2462e950fe7ab0f8d839e9aff1d2"}
Jul 18 00:41:27.470768 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.470753    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:41:27.485242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.485192    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-45txc" podStartSLOduration=66.485175222 podStartE2EDuration="1m6.485175222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:27.484229954 +0000 UTC m=+67.011734913" watchObservedRunningTime="2026-07-18 00:41:27.485175222 +0000 UTC m=+67.012680221"
Jul 18 00:41:27.486113 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.486092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:27.771686 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.771602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:27.839336 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.839296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:27.839336 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.839350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:41:27.839601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.839429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:41:27.839601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.839477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:27.839601 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:27.839489    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 00:41:27.839601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.839541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:41:27.839601 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:27.839565    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert podName:99bcb8a3-01c9-454e-ad27-17bd14fde331 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:59.839543103 +0000 UTC m=+99.367048046 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-sdkv4" (UID: "99bcb8a3-01c9-454e-ad27-17bd14fde331") : secret "networking-console-plugin-cert" not found
Jul 18 00:41:27.839601 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:27.839604    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 00:41:27.839977 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.839623    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:27.839977 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:27.839652    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls podName:7c5c65bb-51cb-4ffa-badb-a0c852c4b366 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:59.839637372 +0000 UTC m=+99.367142310 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-4g6jq" (UID: "7c5c65bb-51cb-4ffa-badb-a0c852c4b366") : secret "cluster-monitoring-operator-tls" not found
Jul 18 00:41:27.840640 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.840255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-service-ca-bundle\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:27.842532 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.842509    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608-metrics-certs\") pod \"router-default-7df7df96f4-vqk8n\" (UID: \"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608\") " pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:27.842645 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.842605    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"image-registry-66945fc9c5-q5mdr\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") " pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:27.842645 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.842638    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/86287ebe-2d1b-4ed6-af4d-622cb20af794-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-zfm7l\" (UID: \"86287ebe-2d1b-4ed6-af4d-622cb20af794\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:41:27.913172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.913147    2567 scope.go:117] "RemoveContainer" containerID="6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c"
Jul 18 00:41:27.940549 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.940512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:27.942938 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.942908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/666e0686-d4c4-4d89-8ff5-e7de233831be-metrics-tls\") pod \"dns-default-f8p9z\" (UID: \"666e0686-d4c4-4d89-8ff5-e7de233831be\") " pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:27.990673 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.990517    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h4ffv\""
Jul 18 00:41:27.999020 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:27.998992    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:28.036960 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.036933    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ctsqr\""
Jul 18 00:41:28.045543 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.044830    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"
Jul 18 00:41:28.075775 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.075622    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bphpb\""
Jul 18 00:41:28.084200 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.084120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:28.109811 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.109782    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9hbbz\""
Jul 18 00:41:28.117254 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.116924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:28.171744 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.170557    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66945fc9c5-q5mdr"]
Jul 18 00:41:28.174790 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:41:28.174758    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb97b25_d83f_4e59_9dfa_5433ca15cbf7.slice/crio-010ba70b28d2f54537c2750aff96b27c7e90c4571a58fa2e0175b048b8665968 WatchSource:0}: Error finding container 010ba70b28d2f54537c2750aff96b27c7e90c4571a58fa2e0175b048b8665968: Status 404 returned error can't find the container with id 010ba70b28d2f54537c2750aff96b27c7e90c4571a58fa2e0175b048b8665968
Jul 18 00:41:28.183487 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.182947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:28.241330 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.238639    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l"]
Jul 18 00:41:28.276016 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.275992    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7df7df96f4-vqk8n"]
Jul 18 00:41:28.281387 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:41:28.280954    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc3c5bf_9d4e_489e_94ae_1ba7fed0a608.slice/crio-9ee263a2c1532e42514cc9bfee8a354cf5ddcf1ea3985a339467e442deb60d94 WatchSource:0}: Error finding container 9ee263a2c1532e42514cc9bfee8a354cf5ddcf1ea3985a339467e442deb60d94: Status 404 returned error can't find the container with id 9ee263a2c1532e42514cc9bfee8a354cf5ddcf1ea3985a339467e442deb60d94
Jul 18 00:41:28.299620 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.299588    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f8p9z"]
Jul 18 00:41:28.302675 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:41:28.302634    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666e0686_d4c4_4d89_8ff5_e7de233831be.slice/crio-b567db8e619623423a37554c6ffa2c24606450301cc8e75cdd25bf44380285b3 WatchSource:0}: Error finding container b567db8e619623423a37554c6ffa2c24606450301cc8e75cdd25bf44380285b3: Status 404 returned error can't find the container with id b567db8e619623423a37554c6ffa2c24606450301cc8e75cdd25bf44380285b3
Jul 18 00:41:28.373557 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.373510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:28.477646 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.477620    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:41:28.478102 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.478080    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/1.log"
Jul 18 00:41:28.478233 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.478124    2567 generic.go:356] "Generic (PLEG): container finished" podID="4dab6681-22ef-4518-9bea-f894457cd197" containerID="ed2599d2798945e191cd9b74633b2f30128ad5e738f8ebaafbac53f6bd22e87b" exitCode=255
Jul 18 00:41:28.478233 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.478175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" event={"ID":"4dab6681-22ef-4518-9bea-f894457cd197","Type":"ContainerDied","Data":"ed2599d2798945e191cd9b74633b2f30128ad5e738f8ebaafbac53f6bd22e87b"}
Jul 18 00:41:28.478408 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.478257    2567 scope.go:117] "RemoveContainer" containerID="6708dcd2a0b8efef3a8be406280ce0e05ebc580aec874d3b2a2e66e39388dd8c"
Jul 18 00:41:28.478651 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.478629    2567 scope.go:117] "RemoveContainer" containerID="ed2599d2798945e191cd9b74633b2f30128ad5e738f8ebaafbac53f6bd22e87b"
Jul 18 00:41:28.478881 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:28.478845    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-rrr5c_openshift-console-operator(4dab6681-22ef-4518-9bea-f894457cd197)\"" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podUID="4dab6681-22ef-4518-9bea-f894457cd197"
Jul 18 00:41:28.479744 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.479718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8p9z" event={"ID":"666e0686-d4c4-4d89-8ff5-e7de233831be","Type":"ContainerStarted","Data":"b567db8e619623423a37554c6ffa2c24606450301cc8e75cdd25bf44380285b3"}
Jul 18 00:41:28.480977 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.480950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l" event={"ID":"86287ebe-2d1b-4ed6-af4d-622cb20af794","Type":"ContainerStarted","Data":"0366ecd583cebe1f1acbec65f9dbe2a368ae58382d65d984a1849917e699cebd"}
Jul 18 00:41:28.482970 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.482525    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7df7df96f4-vqk8n" event={"ID":"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608","Type":"ContainerStarted","Data":"b50e820f6fdabbcb6a60a6d3451d60f7342873f651ae623e4c37bf1095ff1aca"}
Jul 18 00:41:28.482970 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.482567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7df7df96f4-vqk8n" event={"ID":"7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608","Type":"ContainerStarted","Data":"9ee263a2c1532e42514cc9bfee8a354cf5ddcf1ea3985a339467e442deb60d94"}
Jul 18 00:41:28.484340 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.484261    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" event={"ID":"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7","Type":"ContainerStarted","Data":"09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd"}
Jul 18 00:41:28.484340 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.484314    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" event={"ID":"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7","Type":"ContainerStarted","Data":"010ba70b28d2f54537c2750aff96b27c7e90c4571a58fa2e0175b048b8665968"}
Jul 18 00:41:28.512513 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.512440    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7df7df96f4-vqk8n" podStartSLOduration=64.51241816 podStartE2EDuration="1m4.51241816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:28.512372881 +0000 UTC m=+68.039877867" watchObservedRunningTime="2026-07-18 00:41:28.51241816 +0000 UTC m=+68.039923125"
Jul 18 00:41:28.530850 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.530774    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" podStartSLOduration=67.530752122 podStartE2EDuration="1m7.530752122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:28.529165882 +0000 UTC m=+68.056670844" watchObservedRunningTime="2026-07-18 00:41:28.530752122 +0000 UTC m=+68.058257082"
Jul 18 00:41:28.977951 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:28.977916    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:29.084517 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.084481    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:29.087816 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.087613    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:29.349569 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.349482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:29.494783 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.493241    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:41:29.494783 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.494110    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:29.494783 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.494462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:29.495732 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.495541    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7df7df96f4-vqk8n"
Jul 18 00:41:29.579255 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.579219    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:29.775978 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.775899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:29.971905 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:29.971865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:30.395806 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:30.395772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:30.466342 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:30.466297    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:30.466524 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:30.466456    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:30.466524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:30.466480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:30.466524 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:30.466517    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls podName:0e302077-35e3-437a-88ea-a5102042948b nodeName:}" failed. No retries permitted until 2026-07-18 00:41:46.466501404 +0000 UTC m=+85.994006342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7g9c6" (UID: "0e302077-35e3-437a-88ea-a5102042948b") : secret "insights-runtime-extractor-tls" not found
Jul 18 00:41:30.467130 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:30.467109    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e302077-35e3-437a-88ea-a5102042948b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:30.779299 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:30.779183    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:31.383605 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:31.383565    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:31.580720 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:31.580688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:31.779635 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:31.779508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:32.508339 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.508232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4bl8d" event={"ID":"d0098140-e04e-4d1c-b440-09c25f6825f9","Type":"ContainerStarted","Data":"9a35fd5d30ae734280ba8572a12f5917e6c83e2293fc1d255abd815f333eab69"}
Jul 18 00:41:32.510099 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.510069    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l" event={"ID":"86287ebe-2d1b-4ed6-af4d-622cb20af794","Type":"ContainerStarted","Data":"54980ababdf7adf7e897dd2e9612c488ebfe057f495a6f6bb38032cef95a4aa6"}
Jul 18 00:41:32.510254 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.510117    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l" event={"ID":"86287ebe-2d1b-4ed6-af4d-622cb20af794","Type":"ContainerStarted","Data":"6267c76bfbf33e07221c9b4e0887473d5fe53690ce9d7102628cafcb3bbac01d"}
Jul 18 00:41:32.511774 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.511752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8p9z" event={"ID":"666e0686-d4c4-4d89-8ff5-e7de233831be","Type":"ContainerStarted","Data":"7321851b61ee36c3e0a3142a6695a5c9bd1b51b19c773e7193aeba037daa3ff6"}
Jul 18 00:41:32.511910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.511797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8p9z" event={"ID":"666e0686-d4c4-4d89-8ff5-e7de233831be","Type":"ContainerStarted","Data":"7c09532acd5992de85fadf42e07d3550b25b832a974825d86df74b48378859e1"}
Jul 18 00:41:32.511910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.511826    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:32.524087 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.524040    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4bl8d" podStartSLOduration=66.501124936 podStartE2EDuration="1m11.524023901s" totalImagesPullingTime="5.022898965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:32.523488244 +0000 UTC m=+72.050993207" watchObservedRunningTime="2026-07-18 00:41:32.524023901 +0000 UTC m=+72.051528861"
Jul 18 00:41:32.538221 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.538174    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-zfm7l" podStartSLOduration=64.751694495 podStartE2EDuration="1m8.53815992s" totalImagesPullingTime="3.786465425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:32.538076074 +0000 UTC m=+72.065581034" watchObservedRunningTime="2026-07-18 00:41:32.53815992 +0000 UTC m=+72.065664874"
Jul 18 00:41:32.556159 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:32.556110    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f8p9z" podStartSLOduration=33.749217479 podStartE2EDuration="37.556093726s" totalImagesPullingTime="3.806876247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:32.553657299 +0000 UTC m=+72.081162294" watchObservedRunningTime="2026-07-18 00:41:32.556093726 +0000 UTC m=+72.083598685"
Jul 18 00:41:36.251453 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:36.251414    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:41:36.251854 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:36.251475    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:41:36.251854 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:36.251820    2567 scope.go:117] "RemoveContainer" containerID="ed2599d2798945e191cd9b74633b2f30128ad5e738f8ebaafbac53f6bd22e87b"
Jul 18 00:41:36.252008 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:36.251986    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-rrr5c_openshift-console-operator(4dab6681-22ef-4518-9bea-f894457cd197)\"" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podUID="4dab6681-22ef-4518-9bea-f894457cd197"
Jul 18 00:41:39.510571 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:39.510541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:39.600050 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:39.600010    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:39.676171 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:39.676139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:39.775923 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:39.775835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:39.838482 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:39.838450    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:40.045326 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:40.045215    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:40.102197 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:40.102166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:42.219443 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:42.219328    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:42.319875 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:42.319841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:42.518567 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:42.518494    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f8p9z"
Jul 18 00:41:43.569654 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:43.569621    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:43.626939 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:43.626907    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:43.681954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:43.681921    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:46.504895 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:46.504849    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:46.507286 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:46.507241    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e302077-35e3-437a-88ea-a5102042948b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7g9c6\" (UID: \"0e302077-35e3-437a-88ea-a5102042948b\") " pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:46.708224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:46.708184    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w2tbv\""
Jul 18 00:41:46.716198 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:46.716169    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7g9c6"
Jul 18 00:41:46.892107 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:46.892083    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7g9c6"]
Jul 18 00:41:46.894820 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:41:46.894793    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e302077_35e3_437a_88ea_a5102042948b.slice/crio-5fc5738ebbda429c333e1b11567fdd1a5eb72a08e941462aa4fb6daeb4334149 WatchSource:0}: Error finding container 5fc5738ebbda429c333e1b11567fdd1a5eb72a08e941462aa4fb6daeb4334149: Status 404 returned error can't find the container with id 5fc5738ebbda429c333e1b11567fdd1a5eb72a08e941462aa4fb6daeb4334149
Jul 18 00:41:46.914009 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:46.913978    2567 scope.go:117] "RemoveContainer" containerID="ed2599d2798945e191cd9b74633b2f30128ad5e738f8ebaafbac53f6bd22e87b"
Jul 18 00:41:46.914154 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:41:46.914117    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-rrr5c_openshift-console-operator(4dab6681-22ef-4518-9bea-f894457cd197)\"" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podUID="4dab6681-22ef-4518-9bea-f894457cd197"
Jul 18 00:41:47.569614 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:47.569534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g9c6" event={"ID":"0e302077-35e3-437a-88ea-a5102042948b","Type":"ContainerStarted","Data":"5fc5738ebbda429c333e1b11567fdd1a5eb72a08e941462aa4fb6daeb4334149"}
Jul 18 00:41:47.703962 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:47.703928    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:47.766536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:47.766502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:48.004809 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:48.004721    2567 patch_prober.go:28] interesting pod/image-registry-66945fc9c5-q5mdr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 18 00:41:48.004809 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:48.004787    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" podUID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 00:41:50.504413 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:50.504383    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:41:50.798096 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:50.798021    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:50.898968 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:50.898939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:51.647339 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:51.647297    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:51.709045 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:51.709011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:52.592213 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:52.592177    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g9c6" event={"ID":"0e302077-35e3-437a-88ea-a5102042948b","Type":"ContainerStarted","Data":"fc2233ea6b11116a6e59ad8db13d8561f0e495c298f933474edb06bf372c14cd"}
Jul 18 00:41:55.529629 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:55.529473    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:55.606189 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:55.606156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g9c6" event={"ID":"0e302077-35e3-437a-88ea-a5102042948b","Type":"ContainerStarted","Data":"4f7f6277b4c420eb16f66ff4d1c852e92415746a82900e748bf6fc2ed0ab8dba"}
Jul 18 00:41:55.630316 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:55.630262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:58.487362 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:58.487330    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-45txc"
Jul 18 00:41:58.913887 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:58.913862    2567 scope.go:117] "RemoveContainer" containerID="ed2599d2798945e191cd9b74633b2f30128ad5e738f8ebaafbac53f6bd22e87b"
Jul 18 00:41:59.141605 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.141572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:59.246083 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.245990    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:59.624224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.624180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7g9c6" event={"ID":"0e302077-35e3-437a-88ea-a5102042948b","Type":"ContainerStarted","Data":"8ad47fe51c60621d546ae4d78cb981ec659458eecfb3ced9dbe2aca4461d13da"}
Jul 18 00:41:59.624909 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.624878    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:59.626072 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.626054    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:41:59.626152 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.626142    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" event={"ID":"4dab6681-22ef-4518-9bea-f894457cd197","Type":"ContainerStarted","Data":"3aea655faf6d51f0f8e918b993d4362a130086d5863818c0153873ac2550fd8c"}
Jul 18 00:41:59.626368 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.626351    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:41:59.631298 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.631258    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c"
Jul 18 00:41:59.648367 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.648311    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7g9c6" podStartSLOduration=33.77261617 podStartE2EDuration="45.64829277s" totalImagesPullingTime="11.8756766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:41:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:59.647140935 +0000 UTC m=+99.174645896" watchObservedRunningTime="2026-07-18 00:41:59.64829277 +0000 UTC m=+99.175797731"
Jul 18 00:41:59.663867 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.663817    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-rrr5c" podStartSLOduration=81.666274895 podStartE2EDuration="1m35.663799695s" totalImagesPullingTime="13.9975248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:59.662751051 +0000 UTC m=+99.190256012" watchObservedRunningTime="2026-07-18 00:41:59.663799695 +0000 UTC m=+99.191304653"
Jul 18 00:41:59.688184 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.688154    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:59.789228 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.789191    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:59.930544 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.930456    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:41:59.935870 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.935843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:41:59.936006 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.935899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:41:59.938457 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.938436    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99bcb8a3-01c9-454e-ad27-17bd14fde331-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-sdkv4\" (UID: \"99bcb8a3-01c9-454e-ad27-17bd14fde331\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:41:59.938545 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:41:59.938529    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c5c65bb-51cb-4ffa-badb-a0c852c4b366-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-4g6jq\" (UID: \"7c5c65bb-51cb-4ffa-badb-a0c852c4b366\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:42:00.105613 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.105579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:00.130168 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.130137    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8t9b9\""
Jul 18 00:42:00.138323 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.138292    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"
Jul 18 00:42:00.165867 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.165835    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zwp6h\""
Jul 18 00:42:00.174436 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.174409    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"
Jul 18 00:42:00.289002 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.288976    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq"]
Jul 18 00:42:00.291539 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:42:00.291503    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5c65bb_51cb_4ffa_badb_a0c852c4b366.slice/crio-c5f7ce5d8504f3b01cab633a754024c1a0784e37e4cc68eee69ef7c009c100d1 WatchSource:0}: Error finding container c5f7ce5d8504f3b01cab633a754024c1a0784e37e4cc68eee69ef7c009c100d1: Status 404 returned error can't find the container with id c5f7ce5d8504f3b01cab633a754024c1a0784e37e4cc68eee69ef7c009c100d1
Jul 18 00:42:00.330732 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.330706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4"]
Jul 18 00:42:00.333186 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:42:00.333159    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bcb8a3_01c9_454e_ad27_17bd14fde331.slice/crio-095033ecdf8c034cf6595671ccc4977a72fb7f5faccd2a9f8d0290b032c007bc WatchSource:0}: Error finding container 095033ecdf8c034cf6595671ccc4977a72fb7f5faccd2a9f8d0290b032c007bc: Status 404 returned error can't find the container with id 095033ecdf8c034cf6595671ccc4977a72fb7f5faccd2a9f8d0290b032c007bc
Jul 18 00:42:00.511886 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.511790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:00.611756 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.611726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:00.631209 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.631177    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq" event={"ID":"7c5c65bb-51cb-4ffa-badb-a0c852c4b366","Type":"ContainerStarted","Data":"c5f7ce5d8504f3b01cab633a754024c1a0784e37e4cc68eee69ef7c009c100d1"}
Jul 18 00:42:00.632348 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.632326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4" event={"ID":"99bcb8a3-01c9-454e-ad27-17bd14fde331","Type":"ContainerStarted","Data":"095033ecdf8c034cf6595671ccc4977a72fb7f5faccd2a9f8d0290b032c007bc"}
Jul 18 00:42:00.712751 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.712714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:00.927223 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:00.927184    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:01.124144 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.124111    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:01.322024 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.321989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:01.533359 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.533321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:01.639774 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.639173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4" event={"ID":"99bcb8a3-01c9-454e-ad27-17bd14fde331","Type":"ContainerStarted","Data":"e2249abdd9ba88595be2f056964b72be1fb22f9c3c62733df7eb7564e2aaef2e"}
Jul 18 00:42:01.658116 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.658042    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-sdkv4" podStartSLOduration=94.586349195 podStartE2EDuration="1m35.658022608s" totalImagesPullingTime="1.071673413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:42:01.657036471 +0000 UTC m=+101.184541433" watchObservedRunningTime="2026-07-18 00:42:01.658022608 +0000 UTC m=+101.185527572"
Jul 18 00:42:01.743918 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.743887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:01.866118 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.866083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:01.958621 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:01.958544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:02.447171 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.447137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:02.551322 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.551264    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:02.633746 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.633704    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66945fc9c5-q5mdr"]
Jul 18 00:42:02.637527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.637499    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:02.646953 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.646907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq" event={"ID":"7c5c65bb-51cb-4ffa-badb-a0c852c4b366","Type":"ContainerStarted","Data":"47c80387c2d22cc7fb16e42927a0aa89749f1f92853f4f32cd6039d400a39e57"}
Jul 18 00:42:02.684956 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.683089    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-4g6jq" podStartSLOduration=96.858025356 podStartE2EDuration="1m38.683069022s" totalImagesPullingTime="1.825043666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:42:02.680418357 +0000 UTC m=+102.207923318" watchObservedRunningTime="2026-07-18 00:42:02.683069022 +0000 UTC m=+102.210573983"
Jul 18 00:42:02.719467 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.719378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:02.820999 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.820965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:02.923910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:02.923870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:03.127064 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:03.127026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:03.231596 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:03.231556    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:03.397799 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:03.397718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:04.638413 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:04.638378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:04.702129 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:04.702096    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:06.653329 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:06.653292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:06.726773 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:06.726738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:07.055826 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:07.055739    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:07.156437 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:07.156406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:07.258060 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:07.258026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:09.115063 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:09.115027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:09.526051 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:09.525966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:09.621863 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:09.621830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:09.675349 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:09.675308    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:10.043679 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:10.043644    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:10.097111 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:10.097077    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:11.802390 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:11.802360    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:12.022609 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.022570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:12.078496 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.078463    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nsf2c"]
Jul 18 00:42:12.081862 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.081844    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.084192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.084162    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 00:42:12.084361 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.084327    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 00:42:12.084440 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.084383    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hlh57\""
Jul 18 00:42:12.084440 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.084344    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 00:42:12.085005 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.084987    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 00:42:12.106947 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.106915    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:12.180489 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.180452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:12.245161 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-root\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245161 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b412783-2ae1-418a-9f27-42076704b6e5-metrics-client-ca\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245240    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-sys\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245305    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245334    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-wtmp\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245391 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-textfile\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245519 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2l8x\" (UniqueName: \"kubernetes.io/projected/4b412783-2ae1-418a-9f27-42076704b6e5-kube-api-access-p2l8x\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245519 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.245519 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.245485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-tls\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.281699 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.281665    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:12.346601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2l8x\" (UniqueName: \"kubernetes.io/projected/4b412783-2ae1-418a-9f27-42076704b6e5-kube-api-access-p2l8x\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.346601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.346601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-tls\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.346865 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-root\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.346865 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b412783-2ae1-418a-9f27-42076704b6e5-metrics-client-ca\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.346865 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-sys\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.346865 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346822    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-root\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-wtmp\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346894    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-sys\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.346929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-textfile\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.347017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-wtmp\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347342 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.347285    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-textfile\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347342 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.347327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b412783-2ae1-418a-9f27-42076704b6e5-metrics-client-ca\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.347618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.347598    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.349144 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.349114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:12.349259 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.349200    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-tls\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.349439 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.349422    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b412783-2ae1-418a-9f27-42076704b6e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.357505 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.357481    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2l8x\" (UniqueName: \"kubernetes.io/projected/4b412783-2ae1-418a-9f27-42076704b6e5-kube-api-access-p2l8x\") pod \"node-exporter-nsf2c\" (UID: \"4b412783-2ae1-418a-9f27-42076704b6e5\") " pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.391127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.391078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nsf2c"
Jul 18 00:42:12.404354 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:42:12.404318    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b412783_2ae1_418a_9f27_42076704b6e5.slice/crio-4caeedae25f75f2ff975d18a26d783f7fe3c289ac39abd23105959c7caeddd65 WatchSource:0}: Error finding container 4caeedae25f75f2ff975d18a26d783f7fe3c289ac39abd23105959c7caeddd65: Status 404 returned error can't find the container with id 4caeedae25f75f2ff975d18a26d783f7fe3c289ac39abd23105959c7caeddd65
Jul 18 00:42:12.684533 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.684424    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsf2c" event={"ID":"4b412783-2ae1-418a-9f27-42076704b6e5","Type":"ContainerStarted","Data":"4caeedae25f75f2ff975d18a26d783f7fe3c289ac39abd23105959c7caeddd65"}
Jul 18 00:42:12.894503 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:12.894464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:13.033015 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:13.032929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:13.117405 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:13.117306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:13.364306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:13.364248    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:13.690819 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:13.690726    2567 generic.go:356] "Generic (PLEG): container finished" podID="4b412783-2ae1-418a-9f27-42076704b6e5" containerID="c015420b20ebb8f6c06e51c524dd37197735d5aa253b1e2616012acdb4af90ed" exitCode=0
Jul 18 00:42:13.690819 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:13.690781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsf2c" event={"ID":"4b412783-2ae1-418a-9f27-42076704b6e5","Type":"ContainerDied","Data":"c015420b20ebb8f6c06e51c524dd37197735d5aa253b1e2616012acdb4af90ed"}
Jul 18 00:42:14.079812 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:14.079762    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:14.180423 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:14.180389    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:14.696208 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:14.696176    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsf2c" event={"ID":"4b412783-2ae1-418a-9f27-42076704b6e5","Type":"ContainerStarted","Data":"76f0431f1b966a4e0d1426ac156e4dbd0851bca9ce4f124b6ecc977b7f5ee7ed"}
Jul 18 00:42:14.696398 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:14.696226    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsf2c" event={"ID":"4b412783-2ae1-418a-9f27-42076704b6e5","Type":"ContainerStarted","Data":"38adbc0edf34575e4c4bea499ea5a7b51539b66fd00e15d4824caee7eb11fade"}
Jul 18 00:42:14.716527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:14.716472    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nsf2c" podStartSLOduration=1.8353205049999999 podStartE2EDuration="2.716456242s" totalImagesPullingTime="805.9391ms" totalInitContainerRuntime="75.196637ms" isStatefulPod=true podCreationTimestamp="2026-07-18 00:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:42:14.714754316 +0000 UTC m=+114.242259280" watchObservedRunningTime="2026-07-18 00:42:14.716456242 +0000 UTC m=+114.243961203"
Jul 18 00:42:14.990671 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:14.990581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:15.090954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:15.090918    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:15.545153 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:15.545121    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:16.331234 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:16.331200    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:16.399870 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:16.399839    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:16.819955 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:16.819925    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:16.882201 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:16.882171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:17.323723 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:17.323694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:18.230548 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:18.230485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:23.325491 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:23.325457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:23.394594 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:23.394564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:24.735288 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:24.735239    2567 generic.go:356] "Generic (PLEG): container finished" podID="c69309b0-d488-49df-9bab-4cdcb1c74e2f" containerID="a1828a1a878bb2c3c222903e61b8349f7f3c59eba8f4ac4daf9e89a3b7385a3e" exitCode=0
Jul 18 00:42:24.735678 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:24.735289    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf" event={"ID":"c69309b0-d488-49df-9bab-4cdcb1c74e2f","Type":"ContainerDied","Data":"a1828a1a878bb2c3c222903e61b8349f7f3c59eba8f4ac4daf9e89a3b7385a3e"}
Jul 18 00:42:24.735678 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:24.735634    2567 scope.go:117] "RemoveContainer" containerID="a1828a1a878bb2c3c222903e61b8349f7f3c59eba8f4ac4daf9e89a3b7385a3e"
Jul 18 00:42:25.042303 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:25.042211    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:25.142189 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:25.142157    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:25.194634 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:25.194586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:25.741126 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:25.741092    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-lsstf" event={"ID":"c69309b0-d488-49df-9bab-4cdcb1c74e2f","Type":"ContainerStarted","Data":"29ba285169239fab7633240d5def347c966cc38888c9a30991947290efce51c9"}
Jul 18 00:42:27.657038 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.656997    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" podUID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" containerName="registry" containerID="cri-o://09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd" gracePeriod=30
Jul 18 00:42:27.910081 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.910014    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:42:27.984824 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.984781    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-image-registry-private-configuration\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.984824 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.984829    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-installation-pull-secrets\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985043 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.984867    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-trusted-ca\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985043 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.984908    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985139 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985049    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-ca-trust-extracted\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985139 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985108    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-certificates\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985234 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985138    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z4mr\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-kube-api-access-8z4mr\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985234 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985166    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-bound-sa-token\") pod \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\" (UID: \"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7\") "
Jul 18 00:42:27.985395 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985367    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 00:42:27.985653 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985603    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-trusted-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:27.985653 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.985618    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 00:42:27.987518 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.987488    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:42:27.987632 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.987488    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 00:42:27.987700 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.987684    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 00:42:27.988022 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.987988    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-kube-api-access-8z4mr" (OuterVolumeSpecName: "kube-api-access-8z4mr") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "kube-api-access-8z4mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:42:27.988127 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.988062    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:42:27.993669 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:27.993643    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" (UID: "7fb97b25-d83f-4e59-9dfa-5433ca15cbf7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 00:42:28.086072 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086042    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-tls\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.086072 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086069    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-ca-trust-extracted\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.086072 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086079    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-registry-certificates\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.086355 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086089    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z4mr\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-kube-api-access-8z4mr\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.086355 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086097    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-bound-sa-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.086355 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086106    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-image-registry-private-configuration\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.086355 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.086115    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7-installation-pull-secrets\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:42:28.754064 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.754033    2567 generic.go:356] "Generic (PLEG): container finished" podID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" containerID="09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd" exitCode=0
Jul 18 00:42:28.754523 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.754077    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr"
Jul 18 00:42:28.754523 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.754112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" event={"ID":"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7","Type":"ContainerDied","Data":"09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd"}
Jul 18 00:42:28.754523 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.754162    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66945fc9c5-q5mdr" event={"ID":"7fb97b25-d83f-4e59-9dfa-5433ca15cbf7","Type":"ContainerDied","Data":"010ba70b28d2f54537c2750aff96b27c7e90c4571a58fa2e0175b048b8665968"}
Jul 18 00:42:28.754523 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.754185    2567 scope.go:117] "RemoveContainer" containerID="09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd"
Jul 18 00:42:28.762891 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.762873    2567 scope.go:117] "RemoveContainer" containerID="09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd"
Jul 18 00:42:28.763174 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:42:28.763154    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd\": container with ID starting with 09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd not found: ID does not exist" containerID="09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd"
Jul 18 00:42:28.763243 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.763186    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd"} err="failed to get container status \"09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd\": rpc error: code = NotFound desc = could not find container \"09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd\": container with ID starting with 09c7585abba109488b5c7cbf94a638b26bb98a0babe0a6cb728e9db7b444c3bd not found: ID does not exist"
Jul 18 00:42:28.773987 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.773962    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66945fc9c5-q5mdr"]
Jul 18 00:42:28.781986 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.781956    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66945fc9c5-q5mdr"]
Jul 18 00:42:28.918177 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:28.918141    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" path="/var/lib/kubelet/pods/7fb97b25-d83f-4e59-9dfa-5433ca15cbf7/volumes"
Jul 18 00:42:30.807395 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:30.807351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:42:30.809906 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:30.809882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984d0621-5433-4613-b1ff-41f803ca0a53-metrics-certs\") pod \"network-metrics-daemon-qb7rp\" (UID: \"984d0621-5433-4613-b1ff-41f803ca0a53\") " pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:42:30.836088 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:30.836053    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d2hq6\""
Jul 18 00:42:30.843861 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:30.843830    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qb7rp"
Jul 18 00:42:30.979966 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:30.979941    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qb7rp"]
Jul 18 00:42:30.982481 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:42:30.982451    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984d0621_5433_4613_b1ff_41f803ca0a53.slice/crio-5b46ebed1f329487152840434d59d25aa52d979fd4b512ed6e749f803b432fce WatchSource:0}: Error finding container 5b46ebed1f329487152840434d59d25aa52d979fd4b512ed6e749f803b432fce: Status 404 returned error can't find the container with id 5b46ebed1f329487152840434d59d25aa52d979fd4b512ed6e749f803b432fce
Jul 18 00:42:31.768488 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:31.768451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qb7rp" event={"ID":"984d0621-5433-4613-b1ff-41f803ca0a53","Type":"ContainerStarted","Data":"5b46ebed1f329487152840434d59d25aa52d979fd4b512ed6e749f803b432fce"}
Jul 18 00:42:32.774346 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:32.774307    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qb7rp" event={"ID":"984d0621-5433-4613-b1ff-41f803ca0a53","Type":"ContainerStarted","Data":"0565d7a9d40e8f8998548acb6a46fc58d9376f030032b92fb96d16cab116de1f"}
Jul 18 00:42:32.774746 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:32.774375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qb7rp" event={"ID":"984d0621-5433-4613-b1ff-41f803ca0a53","Type":"ContainerStarted","Data":"72882be0951eb22c321c5686535f84592c7aac1285fbe3785eca7ee5f4691192"}
Jul 18 00:42:32.792888 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:32.792805    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qb7rp" podStartSLOduration=130.832097372 podStartE2EDuration="2m11.792784133s" totalImagesPullingTime="960.686761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:42:32.791014018 +0000 UTC m=+132.318518979" watchObservedRunningTime="2026-07-18 00:42:32.792784133 +0000 UTC m=+132.320289122"
Jul 18 00:42:32.882125 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:32.882071    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:32.937759 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:32.937725    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:33.534280 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:33.534244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:33.617519 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:33.617485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:34.134837 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:34.134801    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:34.183457 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:34.183379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:34.783594 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:34.783561    2567 generic.go:356] "Generic (PLEG): container finished" podID="c4238d52-036a-4948-88cd-3d2f814d369c" containerID="fb39c8eedcd2a7a79ff3261d0cb747adc189a332b91b9854e9fda11d942e018e" exitCode=0
Jul 18 00:42:34.783594 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:34.783598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-km4hb" event={"ID":"c4238d52-036a-4948-88cd-3d2f814d369c","Type":"ContainerDied","Data":"fb39c8eedcd2a7a79ff3261d0cb747adc189a332b91b9854e9fda11d942e018e"}
Jul 18 00:42:34.783925 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:34.783912    2567 scope.go:117] "RemoveContainer" containerID="fb39c8eedcd2a7a79ff3261d0cb747adc189a332b91b9854e9fda11d942e018e"
Jul 18 00:42:35.789691 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:35.789656    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-km4hb" event={"ID":"c4238d52-036a-4948-88cd-3d2f814d369c","Type":"ContainerStarted","Data":"fbc03058a8c50eddee2093a3b67d21c4e62d95ea3b65227ad5cc35b0d8234875"}
Jul 18 00:42:37.419165 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:37.419125    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" podUID="8f243a49-0287-415e-89b8-2201b0a9a6c1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 18 00:42:40.041230 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:40.041184    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:40.093208 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:40.093172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:42:43.819988 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:43.819959    2567 generic.go:356] "Generic (PLEG): container finished" podID="b36a62f2-c75b-4a7c-97f4-e9dd95ee4254" containerID="93b8488842f7e85d24603fd081f73f780768426754b4c5f43aabb396834ff8ce" exitCode=0
Jul 18 00:42:43.820431 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:43.820033    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f" event={"ID":"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254","Type":"ContainerDied","Data":"93b8488842f7e85d24603fd081f73f780768426754b4c5f43aabb396834ff8ce"}
Jul 18 00:42:43.820431 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:43.820358    2567 scope.go:117] "RemoveContainer" containerID="93b8488842f7e85d24603fd081f73f780768426754b4c5f43aabb396834ff8ce"
Jul 18 00:42:44.825537 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:44.825507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-klc2f" event={"ID":"b36a62f2-c75b-4a7c-97f4-e9dd95ee4254","Type":"ContainerStarted","Data":"cd22a65ec2e4a98f18afab6bfa08507eb367569a02f643bbad81d215caf473e6"}
Jul 18 00:42:47.419478 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:47.419435    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" podUID="8f243a49-0287-415e-89b8-2201b0a9a6c1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 18 00:42:57.419172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.419093    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" podUID="8f243a49-0287-415e-89b8-2201b0a9a6c1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 18 00:42:57.419172 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.419174    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7"
Jul 18 00:42:57.419832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.419655    2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"06e9b35a1687f5a8eafffe80bc893a6d953955b5a86a419d76a08124077701ba"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 18 00:42:57.419832 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.419693    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" podUID="8f243a49-0287-415e-89b8-2201b0a9a6c1" containerName="service-proxy" containerID="cri-o://06e9b35a1687f5a8eafffe80bc893a6d953955b5a86a419d76a08124077701ba" gracePeriod=30
Jul 18 00:42:57.881752 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.881717    2567 generic.go:356] "Generic (PLEG): container finished" podID="8f243a49-0287-415e-89b8-2201b0a9a6c1" containerID="06e9b35a1687f5a8eafffe80bc893a6d953955b5a86a419d76a08124077701ba" exitCode=2
Jul 18 00:42:57.881926 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.881794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" event={"ID":"8f243a49-0287-415e-89b8-2201b0a9a6c1","Type":"ContainerDied","Data":"06e9b35a1687f5a8eafffe80bc893a6d953955b5a86a419d76a08124077701ba"}
Jul 18 00:42:57.881926 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:42:57.881839    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bd58b68d5-jdxr7" event={"ID":"8f243a49-0287-415e-89b8-2201b0a9a6c1","Type":"ContainerStarted","Data":"0652934316a59978b2ad1e1da3deacbb6753e80f3179f7c9765d94d78c9339bb"}
Jul 18 00:43:31.247640 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:31.247597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:31.686397 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:31.686364    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:40.941849 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:40.941762    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:41.086170 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:41.086131    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:41.138676 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:41.138638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:41.687211 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:41.687170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:41.745619 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:41.745592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:52.192687 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:52.192630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:52.293565 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:52.293527    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:43:52.346654 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:43:52.346624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:05.799779 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:05.799749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:05.901465 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:05.901422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:46.506981 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:46.506897    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:46.557393 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:46.557355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:47.113421 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:47.113383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:47.175139 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:47.175109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:47.247219 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:47.247192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:47.293367 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:47.293331    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:47.855138 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:47.855108    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:47.904570 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:47.904542    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:58.426376 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:58.426342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:44:59.339524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:44:59.339488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:10.055671 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:10.055630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:10.156442 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:10.156409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:20.816565 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:20.816540    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:45:20.822527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:20.822499    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:45:20.828870 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:20.828850    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 00:45:34.071602 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:34.071565    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:34.172111 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:34.172078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:44.408156 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:44.408114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:55.836985 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:55.836949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:57.219977 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:57.219945    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:59.444785 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:59.444732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:45:59.545848 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:45:59.545806    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:00.931431 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:00.931378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:01.057854 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:01.057814    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:01.781773 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:01.781744    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:01.935187 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:01.935154    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:04.196286 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:04.196189    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:04.348802 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:04.348772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:06.447830 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:06.447788    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:06.527671 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:06.527633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:10.071837 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:10.071802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:10.172767 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:10.172732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:11.757143 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:11.757060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:18.057001 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:18.056965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:18.156999 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:18.156962    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:23.285460 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:23.285418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:23.385251 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:23.385212    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:25.120502 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:25.120469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:38.114658 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:38.114622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:38.201535 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:38.201496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:38.688748 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:38.688716    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:38.993371 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:38.993294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:39.092466 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.092431    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:39.206788 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.206756    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"]
Jul 18 00:46:39.207167 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.207048    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" containerName="registry"
Jul 18 00:46:39.207167 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.207062    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" containerName="registry"
Jul 18 00:46:39.207167 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.207120    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb97b25-d83f-4e59-9dfa-5433ca15cbf7" containerName="registry"
Jul 18 00:46:39.210100 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.210084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.226112 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.226084    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 00:46:39.226308 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.226085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jk4rp\""
Jul 18 00:46:39.226308 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.226133    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 00:46:39.226308 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.226085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 00:46:39.226308 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.226121    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 00:46:39.228821 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.228801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3611f732-9623-40c4-bcbe-7bdbd511d053-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.228903 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.228831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phbvk\" (UniqueName: \"kubernetes.io/projected/3611f732-9623-40c4-bcbe-7bdbd511d053-kube-api-access-phbvk\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.228903 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.228854    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3611f732-9623-40c4-bcbe-7bdbd511d053-manifests\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.228983 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.228930    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3611f732-9623-40c4-bcbe-7bdbd511d053-tmp\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.229024 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.228988    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3611f732-9623-40c4-bcbe-7bdbd511d053-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.270736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.270653    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"]
Jul 18 00:46:39.329937 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.329900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3611f732-9623-40c4-bcbe-7bdbd511d053-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.329937 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.329939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phbvk\" (UniqueName: \"kubernetes.io/projected/3611f732-9623-40c4-bcbe-7bdbd511d053-kube-api-access-phbvk\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.330156 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.329961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3611f732-9623-40c4-bcbe-7bdbd511d053-manifests\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.330156 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.330008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3611f732-9623-40c4-bcbe-7bdbd511d053-tmp\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.330257 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.330236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3611f732-9623-40c4-bcbe-7bdbd511d053-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.330426 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.330410    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3611f732-9623-40c4-bcbe-7bdbd511d053-manifests\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.332574 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.332548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3611f732-9623-40c4-bcbe-7bdbd511d053-tmp\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.332699 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.332614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3611f732-9623-40c4-bcbe-7bdbd511d053-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.332699 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.332656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3611f732-9623-40c4-bcbe-7bdbd511d053-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.353238 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.353206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phbvk\" (UniqueName: \"kubernetes.io/projected/3611f732-9623-40c4-bcbe-7bdbd511d053-kube-api-access-phbvk\") pod \"opendatahub-operator-controller-manager-6bd8c7c4c4-9n688\" (UID: \"3611f732-9623-40c4-bcbe-7bdbd511d053\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.435562 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.435530    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:39.519663 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.519630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:39.527111 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.527086    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:39.628937 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.628904    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:39.667543 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.667519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"]
Jul 18 00:46:39.670096 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:46:39.670069    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3611f732_9623_40c4_bcbe_7bdbd511d053.slice/crio-4791f654901bab5e2c42d914d0dcccaad3010e1a38b36207296f1d9c8c674e87 WatchSource:0}: Error finding container 4791f654901bab5e2c42d914d0dcccaad3010e1a38b36207296f1d9c8c674e87: Status 404 returned error can't find the container with id 4791f654901bab5e2c42d914d0dcccaad3010e1a38b36207296f1d9c8c674e87
Jul 18 00:46:39.671808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.671792    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 00:46:39.707736 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.707702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688" event={"ID":"3611f732-9623-40c4-bcbe-7bdbd511d053","Type":"ContainerStarted","Data":"4791f654901bab5e2c42d914d0dcccaad3010e1a38b36207296f1d9c8c674e87"}
Jul 18 00:46:39.880972 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.880933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:39.972935 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:39.972898    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:42.725019 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:42.724982    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688" event={"ID":"3611f732-9623-40c4-bcbe-7bdbd511d053","Type":"ContainerStarted","Data":"8aa619c6587b0b91c34093ed82428c81171c6070c56c495f4f4be323024fb898"}
Jul 18 00:46:43.731095 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:43.731059    2567 generic.go:356] "Generic (PLEG): container finished" podID="3611f732-9623-40c4-bcbe-7bdbd511d053" containerID="8aa619c6587b0b91c34093ed82428c81171c6070c56c495f4f4be323024fb898" exitCode=0
Jul 18 00:46:43.731516 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:43.731151    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688" event={"ID":"3611f732-9623-40c4-bcbe-7bdbd511d053","Type":"ContainerDied","Data":"8aa619c6587b0b91c34093ed82428c81171c6070c56c495f4f4be323024fb898"}
Jul 18 00:46:44.737897 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:44.737857    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688" event={"ID":"3611f732-9623-40c4-bcbe-7bdbd511d053","Type":"ContainerStarted","Data":"64939593484c6bfc809a3961dbff3a5d498f6ff95d000fc5bbbd3d637c6b6c34"}
Jul 18 00:46:44.738335 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:44.738002    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:44.763194 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:44.763126    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688" podStartSLOduration=1.5162180859999999 podStartE2EDuration="5.763102818s" totalImagesPullingTime="3.150201155s" totalInitContainerRuntime="1.096683577s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:46:44.761420132 +0000 UTC m=+384.288925105" watchObservedRunningTime="2026-07-18 00:46:44.763102818 +0000 UTC m=+384.290607778"
Jul 18 00:46:51.295029 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:51.294998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:51.389315 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:51.389260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:54.103507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:54.103470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:54.222778 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:54.222743    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:55.744371 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:55.744340    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd8c7c4c4-9n688"
Jul 18 00:46:58.514183 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:58.514137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:58.657090 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:58.657057    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:59.407328 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.407296    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:59.474546 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.474503    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gpxfc"]
Jul 18 00:46:59.488891 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.488859    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:46:59.497195 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.497164    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 18 00:46:59.499167 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.499142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dsgtb\""
Jul 18 00:46:59.511548 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.511519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gpxfc"]
Jul 18 00:46:59.580748 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.580716    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"]
Jul 18 00:46:59.584386 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.584364    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.586856 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.586828    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 00:46:59.587001 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.586922    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 00:46:59.587001 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.586934    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l5n8l\""
Jul 18 00:46:59.603208 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.603173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdpn5\" (UniqueName: \"kubernetes.io/projected/66288b5c-9cce-42a9-a9ec-7132892da61e-kube-api-access-vdpn5\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:46:59.603560 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.603538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66288b5c-9cce-42a9-a9ec-7132892da61e-cert\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:46:59.604864 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.604843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"]
Jul 18 00:46:59.628783 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.628752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:59.704443 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.704356    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66288b5c-9cce-42a9-a9ec-7132892da61e-cert\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:46:59.704443 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.704400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88slh\" (UniqueName: \"kubernetes.io/projected/24c8026f-5014-4d63-95a0-5d46060e2ecc-kube-api-access-88slh\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.704443 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.704431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24c8026f-5014-4d63-95a0-5d46060e2ecc-tls-certs\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.704663 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:46:59.704524    2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 18 00:46:59.704663 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.704553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdpn5\" (UniqueName: \"kubernetes.io/projected/66288b5c-9cce-42a9-a9ec-7132892da61e-kube-api-access-vdpn5\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:46:59.704663 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.704573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c8026f-5014-4d63-95a0-5d46060e2ecc-tmp\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.704663 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:46:59.704601    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66288b5c-9cce-42a9-a9ec-7132892da61e-cert podName:66288b5c-9cce-42a9-a9ec-7132892da61e nodeName:}" failed. No retries permitted until 2026-07-18 00:47:00.204583841 +0000 UTC m=+399.732088779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/66288b5c-9cce-42a9-a9ec-7132892da61e-cert") pod "odh-model-controller-7998c67dc4-gpxfc" (UID: "66288b5c-9cce-42a9-a9ec-7132892da61e") : secret "odh-model-controller-webhook-cert" not found
Jul 18 00:46:59.727405 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.727380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdpn5\" (UniqueName: \"kubernetes.io/projected/66288b5c-9cce-42a9-a9ec-7132892da61e-kube-api-access-vdpn5\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:46:59.805616 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.805574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88slh\" (UniqueName: \"kubernetes.io/projected/24c8026f-5014-4d63-95a0-5d46060e2ecc-kube-api-access-88slh\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.805801 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.805626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24c8026f-5014-4d63-95a0-5d46060e2ecc-tls-certs\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.805801 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.805654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c8026f-5014-4d63-95a0-5d46060e2ecc-tmp\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.808214 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.808187    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c8026f-5014-4d63-95a0-5d46060e2ecc-tmp\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.808471 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.808451    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24c8026f-5014-4d63-95a0-5d46060e2ecc-tls-certs\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.828457 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.828432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88slh\" (UniqueName: \"kubernetes.io/projected/24c8026f-5014-4d63-95a0-5d46060e2ecc-kube-api-access-88slh\") pod \"kube-auth-proxy-7569c6bc9b-vmpgv\" (UID: \"24c8026f-5014-4d63-95a0-5d46060e2ecc\") " pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.846328 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.846293    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:46:59.894373 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.894332    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"
Jul 18 00:46:59.997137 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:46:59.997100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:00.049588 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.049566    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv"]
Jul 18 00:47:00.052168 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:47:00.052136    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c8026f_5014_4d63_95a0_5d46060e2ecc.slice/crio-20c8cde2abe0caafd6c592e66b76320877d5ed49bf5f30118ef715a392b3e058 WatchSource:0}: Error finding container 20c8cde2abe0caafd6c592e66b76320877d5ed49bf5f30118ef715a392b3e058: Status 404 returned error can't find the container with id 20c8cde2abe0caafd6c592e66b76320877d5ed49bf5f30118ef715a392b3e058
Jul 18 00:47:00.210549 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.210521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66288b5c-9cce-42a9-a9ec-7132892da61e-cert\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:47:00.212977 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.212947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66288b5c-9cce-42a9-a9ec-7132892da61e-cert\") pod \"odh-model-controller-7998c67dc4-gpxfc\" (UID: \"66288b5c-9cce-42a9-a9ec-7132892da61e\") " pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:47:00.402062 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.402024    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:47:00.560502 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.560378    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gpxfc"]
Jul 18 00:47:00.562833 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:47:00.562808    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66288b5c_9cce_42a9_a9ec_7132892da61e.slice/crio-2fa0f0459a3501b3427a7b4db14b0fae8513caec0aa2e9bcddb833b2f8ceb0fd WatchSource:0}: Error finding container 2fa0f0459a3501b3427a7b4db14b0fae8513caec0aa2e9bcddb833b2f8ceb0fd: Status 404 returned error can't find the container with id 2fa0f0459a3501b3427a7b4db14b0fae8513caec0aa2e9bcddb833b2f8ceb0fd
Jul 18 00:47:00.733746 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.733659    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lkhmr"]
Jul 18 00:47:00.737215 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.737181    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.746844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.746670    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 18 00:47:00.770116 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.770078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:00.804175 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.804140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv" event={"ID":"24c8026f-5014-4d63-95a0-5d46060e2ecc","Type":"ContainerStarted","Data":"20c8cde2abe0caafd6c592e66b76320877d5ed49bf5f30118ef715a392b3e058"}
Jul 18 00:47:00.805546 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.805510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc" event={"ID":"66288b5c-9cce-42a9-a9ec-7132892da61e","Type":"ContainerStarted","Data":"2fa0f0459a3501b3427a7b4db14b0fae8513caec0aa2e9bcddb833b2f8ceb0fd"}
Jul 18 00:47:00.816069 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.816033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg7rd\" (UniqueName: \"kubernetes.io/projected/12dc7c62-a256-4a17-bee1-ed09820e3a01-kube-api-access-hg7rd\") pod \"model-serving-api-54d89bd5fc-lkhmr\" (UID: \"12dc7c62-a256-4a17-bee1-ed09820e3a01\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.816222 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.816090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12dc7c62-a256-4a17-bee1-ed09820e3a01-tls-certs\") pod \"model-serving-api-54d89bd5fc-lkhmr\" (UID: \"12dc7c62-a256-4a17-bee1-ed09820e3a01\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.848845 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.848765    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lkhmr"]
Jul 18 00:47:00.917590 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.917458    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12dc7c62-a256-4a17-bee1-ed09820e3a01-tls-certs\") pod \"model-serving-api-54d89bd5fc-lkhmr\" (UID: \"12dc7c62-a256-4a17-bee1-ed09820e3a01\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.917779 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.917723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7rd\" (UniqueName: \"kubernetes.io/projected/12dc7c62-a256-4a17-bee1-ed09820e3a01-kube-api-access-hg7rd\") pod \"model-serving-api-54d89bd5fc-lkhmr\" (UID: \"12dc7c62-a256-4a17-bee1-ed09820e3a01\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.924712 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.924686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12dc7c62-a256-4a17-bee1-ed09820e3a01-tls-certs\") pod \"model-serving-api-54d89bd5fc-lkhmr\" (UID: \"12dc7c62-a256-4a17-bee1-ed09820e3a01\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.933639 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.933613    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7rd\" (UniqueName: \"kubernetes.io/projected/12dc7c62-a256-4a17-bee1-ed09820e3a01-kube-api-access-hg7rd\") pod \"model-serving-api-54d89bd5fc-lkhmr\" (UID: \"12dc7c62-a256-4a17-bee1-ed09820e3a01\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:00.995858 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:00.995767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:01.052851 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:01.052817    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:01.261850 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:01.259920    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lkhmr"]
Jul 18 00:47:01.814102 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:01.814064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr" event={"ID":"12dc7c62-a256-4a17-bee1-ed09820e3a01","Type":"ContainerStarted","Data":"55bc51709997c562de747efafd130d96a96b695429a74563017cd07ac876b90b"}
Jul 18 00:47:02.128342 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.126629    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"]
Jul 18 00:47:02.132203 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.130970    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.134186 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.134000    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 18 00:47:02.138586 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.137334    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 18 00:47:02.138586 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.137545    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 18 00:47:02.138586 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.138411    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-df2vt\""
Jul 18 00:47:02.144008 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.143329    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 18 00:47:02.159138 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.159104    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:02.231436 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-token\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.231643 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.231643 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-cacerts\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.231643 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.231643 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231619    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4swc9\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-kube-api-access-4swc9\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.231643 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/511ee99a-b253-42a7-8fdb-bedc95ec369c-local-certs\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.231954 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.231677    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.235560 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.235529    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"]
Jul 18 00:47:02.332253 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4swc9\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-kube-api-access-4swc9\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/511ee99a-b253-42a7-8fdb-bedc95ec369c-local-certs\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-token\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332419 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-cacerts\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.332978 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.332925    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.335508 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.335473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/511ee99a-b253-42a7-8fdb-bedc95ec369c-local-certs\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.335508 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.335489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-cacerts\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.335680 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.335666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.335771 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.335754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.339547 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.339523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:02.342607 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.342582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-token\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.345022 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.345001    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4swc9\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-kube-api-access-4swc9\") pod \"istiod-openshift-gateway-7574459656-9nrdt\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:02.448577 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:02.448493    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:03.591134 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:03.591100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:04.654081 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.654053    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"]
Jul 18 00:47:04.657321 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:47:04.657285    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511ee99a_b253_42a7_8fdb_bedc95ec369c.slice/crio-71db982cfd67c37f2087e1a501af8ce751c119cb4359bd098385e0039f45b2ca WatchSource:0}: Error finding container 71db982cfd67c37f2087e1a501af8ce751c119cb4359bd098385e0039f45b2ca: Status 404 returned error can't find the container with id 71db982cfd67c37f2087e1a501af8ce751c119cb4359bd098385e0039f45b2ca
Jul 18 00:47:04.831655 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.831547    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc" event={"ID":"66288b5c-9cce-42a9-a9ec-7132892da61e","Type":"ContainerStarted","Data":"5c3d6b50324289c045ac5ebcd54d71c46717b9975c2fdd084672aec496c79c42"}
Jul 18 00:47:04.833450 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.833406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" event={"ID":"511ee99a-b253-42a7-8fdb-bedc95ec369c","Type":"ContainerStarted","Data":"71db982cfd67c37f2087e1a501af8ce751c119cb4359bd098385e0039f45b2ca"}
Jul 18 00:47:04.835468 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.835428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr" event={"ID":"12dc7c62-a256-4a17-bee1-ed09820e3a01","Type":"ContainerStarted","Data":"1d61a05005ab89a6db006a8233efc9960bed7ca6b159df76bb3e66c77643784c"}
Jul 18 00:47:04.835799 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.835779    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:04.837567 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.837516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv" event={"ID":"24c8026f-5014-4d63-95a0-5d46060e2ecc","Type":"ContainerStarted","Data":"ee09ea539eb3c3a0c8b194ff83d6eb9a976a3536532ff7340873e25074223c63"}
Jul 18 00:47:04.849961 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.849915    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc" podStartSLOduration=1.922481253 podStartE2EDuration="5.849901531s" totalImagesPullingTime="3.927420278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:04.848979067 +0000 UTC m=+404.376484028" watchObservedRunningTime="2026-07-18 00:47:04.849901531 +0000 UTC m=+404.377406483"
Jul 18 00:47:04.868841 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.868772    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7569c6bc9b-vmpgv" podStartSLOduration=1.444782767 podStartE2EDuration="5.868749189s" totalImagesPullingTime="4.423966422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:04.866100224 +0000 UTC m=+404.393605185" watchObservedRunningTime="2026-07-18 00:47:04.868749189 +0000 UTC m=+404.396254150"
Jul 18 00:47:04.886054 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:04.885977    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr" podStartSLOduration=1.679730832 podStartE2EDuration="4.885953345s" totalImagesPullingTime="3.206222513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:47:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:04.884309837 +0000 UTC m=+404.411814799" watchObservedRunningTime="2026-07-18 00:47:04.885953345 +0000 UTC m=+404.413458306"
Jul 18 00:47:05.846958 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:05.846928    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:47:07.286040 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:07.285985    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:47:07.286375 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:07.286085    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:47:07.858911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:07.858878    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" event={"ID":"511ee99a-b253-42a7-8fdb-bedc95ec369c","Type":"ContainerStarted","Data":"eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02"}
Jul 18 00:47:07.859089 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:07.859011    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:07.907799 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:07.907740    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" podStartSLOduration=3.282088157 podStartE2EDuration="5.907723577s" totalImagesPullingTime="2.62563542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:07.90573664 +0000 UTC m=+407.433241602" watchObservedRunningTime="2026-07-18 00:47:07.907723577 +0000 UTC m=+407.435228537"
Jul 18 00:47:08.218401 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:08.218294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:08.319214 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:08.319183    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:08.865633 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:08.865601    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-9nrdt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 00:47:08.865853 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:08.865663    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" podUID="511ee99a-b253-42a7-8fdb-bedc95ec369c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 00:47:09.277741 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.277653    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"]
Jul 18 00:47:09.281134 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.281116    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.283738 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.283720    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mtvnt\""
Jul 18 00:47:09.292509 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.292484    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"]
Jul 18 00:47:09.318703 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.318663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:09.395616 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395733    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395763    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.395973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-885hq\" (UniqueName: \"kubernetes.io/projected/76d97c25-5c24-4f78-92a8-e1c87e2f0298-kube-api-access-885hq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.396030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.396002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496480 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496522    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-885hq\" (UniqueName: \"kubernetes.io/projected/76d97c25-5c24-4f78-92a8-e1c87e2f0298-kube-api-access-885hq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496626 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496884 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496884 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.496884 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496820    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.497029 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.497029 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.496979    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.497107 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.497067    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.497180 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.497158    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.497507 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.497491    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.498857 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.498835    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.499199 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.499182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.511154 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.511132    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76d97c25-5c24-4f78-92a8-e1c87e2f0298-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.517670 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.517647    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-885hq\" (UniqueName: \"kubernetes.io/projected/76d97c25-5c24-4f78-92a8-e1c87e2f0298-kube-api-access-885hq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qgn9p\" (UID: \"76d97c25-5c24-4f78-92a8-e1c87e2f0298\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.588761 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.588716    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:09.591684 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.591660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:09.728767 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.728739    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:09.743489 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.743465    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"]
Jul 18 00:47:09.745673 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:47:09.745645    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d97c25_5c24_4f78_92a8_e1c87e2f0298.slice/crio-92a924605a09ded14dce6f51036e8c743e1fc7eeece04ec17b28e9acd09186ad WatchSource:0}: Error finding container 92a924605a09ded14dce6f51036e8c743e1fc7eeece04ec17b28e9acd09186ad: Status 404 returned error can't find the container with id 92a924605a09ded14dce6f51036e8c743e1fc7eeece04ec17b28e9acd09186ad
Jul 18 00:47:09.869554 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:09.869474    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p" event={"ID":"76d97c25-5c24-4f78-92a8-e1c87e2f0298","Type":"ContainerStarted","Data":"92a924605a09ded14dce6f51036e8c743e1fc7eeece04ec17b28e9acd09186ad"}
Jul 18 00:47:11.169669 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.169623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:11.235478 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.235441    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"]
Jul 18 00:47:11.239315 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.239291    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.241961 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.241935    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 18 00:47:11.242088 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.241973    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vjm5b\""
Jul 18 00:47:11.252112 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.252081    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"]
Jul 18 00:47:11.290623 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.290594    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:11.313655 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.313611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.313869 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.313725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwvr9\" (UniqueName: \"kubernetes.io/projected/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-kube-api-access-vwvr9\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.414695 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.414643    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwvr9\" (UniqueName: \"kubernetes.io/projected/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-kube-api-access-vwvr9\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.414885 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.414748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.414948 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:47:11.414921    2567 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 18 00:47:11.415015 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:47:11.415002    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-cert podName:0972432b-1e47-4eea-9ba5-5d0ab0d94b55 nodeName:}" failed. No retries permitted until 2026-07-18 00:47:11.914981304 +0000 UTC m=+411.442486244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-cert") pod "kserve-controller-manager-d6f4b65b8-z8dkt" (UID: "0972432b-1e47-4eea-9ba5-5d0ab0d94b55") : secret "kserve-webhook-server-cert" not found
Jul 18 00:47:11.439965 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.439885    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwvr9\" (UniqueName: \"kubernetes.io/projected/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-kube-api-access-vwvr9\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.532479 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.532442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:11.651893 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.651841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:11.853642 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.853596    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:11.865422 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.865397    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:11.919979 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.919946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.923153 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.923117    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0972432b-1e47-4eea-9ba5-5d0ab0d94b55-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8dkt\" (UID: \"0972432b-1e47-4eea-9ba5-5d0ab0d94b55\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:11.949442 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:11.949410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:12.054909 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.054868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:12.153517 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.153438    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:12.414282 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.414227    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:47:12.414574 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.414355    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:47:12.414574 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.414411    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:47:12.484641 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.484623    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"]
Jul 18 00:47:12.487055 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:47:12.487024    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0972432b_1e47_4eea_9ba5_5d0ab0d94b55.slice/crio-1ba6658942524e5180debba8cc66ed66c8abe29cffe351cf1cd916c9881e9a93 WatchSource:0}: Error finding container 1ba6658942524e5180debba8cc66ed66c8abe29cffe351cf1cd916c9881e9a93: Status 404 returned error can't find the container with id 1ba6658942524e5180debba8cc66ed66c8abe29cffe351cf1cd916c9881e9a93
Jul 18 00:47:12.884694 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.884660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p" event={"ID":"76d97c25-5c24-4f78-92a8-e1c87e2f0298","Type":"ContainerStarted","Data":"10ec2c505650d6007d113db179e19af2598e83f5bbdfb940f9ca5ec58747b53a"}
Jul 18 00:47:12.886170 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.886141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt" event={"ID":"0972432b-1e47-4eea-9ba5-5d0ab0d94b55","Type":"ContainerStarted","Data":"1ba6658942524e5180debba8cc66ed66c8abe29cffe351cf1cd916c9881e9a93"}
Jul 18 00:47:12.905911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.905861    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p" podStartSLOduration=1.239663806 podStartE2EDuration="3.905843811s" totalImagesPullingTime="2.666180005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:12.905047593 +0000 UTC m=+412.432552549" watchObservedRunningTime="2026-07-18 00:47:12.905843811 +0000 UTC m=+412.433348771"
Jul 18 00:47:12.921370 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.921340    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"]
Jul 18 00:47:12.921528 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:12.921508    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" podUID="511ee99a-b253-42a7-8fdb-bedc95ec369c" containerName="discovery" containerID="cri-o://eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02" gracePeriod=30
Jul 18 00:47:13.186585 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.186556    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:13.333395 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333355    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-kubeconfig\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.333558 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333407    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-cacerts\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.333612 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333593    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-ca-configmap\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.333666 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333636    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-token\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.333717 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333669    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-dns-cert\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.333717 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333702    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/511ee99a-b253-42a7-8fdb-bedc95ec369c-local-certs\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.333826 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333775    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4swc9\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-kube-api-access-4swc9\") pod \"511ee99a-b253-42a7-8fdb-bedc95ec369c\" (UID: \"511ee99a-b253-42a7-8fdb-bedc95ec369c\") "
Jul 18 00:47:13.334142 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.333963    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 00:47:13.334142 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.334054    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-ca-configmap\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.336735 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.336704    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-cacerts" (OuterVolumeSpecName: "cacerts") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 00:47:13.336851 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.336768    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 00:47:13.336920 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.336855    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-kube-api-access-4swc9" (OuterVolumeSpecName: "kube-api-access-4swc9") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "kube-api-access-4swc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:47:13.337112 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.337086    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-token" (OuterVolumeSpecName: "istio-token") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:47:13.337175 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.337134    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 00:47:13.337506 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.337481    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511ee99a-b253-42a7-8fdb-bedc95ec369c-local-certs" (OuterVolumeSpecName: "local-certs") pod "511ee99a-b253-42a7-8fdb-bedc95ec369c" (UID: "511ee99a-b253-42a7-8fdb-bedc95ec369c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 00:47:13.435378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.435294    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.435378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.435331    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-csr-dns-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.435378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.435343    2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/511ee99a-b253-42a7-8fdb-bedc95ec369c-local-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.435378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.435352    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4swc9\" (UniqueName: \"kubernetes.io/projected/511ee99a-b253-42a7-8fdb-bedc95ec369c-kube-api-access-4swc9\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.435378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.435365    2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-istio-kubeconfig\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.435378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.435378    2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/511ee99a-b253-42a7-8fdb-bedc95ec369c-cacerts\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:47:13.592550 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.592505    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:13.597708 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.597674    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:13.893282 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.893227    2567 generic.go:356] "Generic (PLEG): container finished" podID="511ee99a-b253-42a7-8fdb-bedc95ec369c" containerID="eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02" exitCode=0
Jul 18 00:47:13.893487 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.893304    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"
Jul 18 00:47:13.893487 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.893303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" event={"ID":"511ee99a-b253-42a7-8fdb-bedc95ec369c","Type":"ContainerDied","Data":"eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02"}
Jul 18 00:47:13.893487 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.893359    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt" event={"ID":"511ee99a-b253-42a7-8fdb-bedc95ec369c","Type":"ContainerDied","Data":"71db982cfd67c37f2087e1a501af8ce751c119cb4359bd098385e0039f45b2ca"}
Jul 18 00:47:13.893487 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.893387    2567 scope.go:117] "RemoveContainer" containerID="eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02"
Jul 18 00:47:13.894072 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.894049    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:13.895209 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.895161    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qgn9p"
Jul 18 00:47:13.903030 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.903014    2567 scope.go:117] "RemoveContainer" containerID="eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02"
Jul 18 00:47:13.903353 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:47:13.903336    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02\": container with ID starting with eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02 not found: ID does not exist" containerID="eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02"
Jul 18 00:47:13.903412 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.903367    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02"} err="failed to get container status \"eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02\": rpc error: code = NotFound desc = could not find container \"eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02\": container with ID starting with eb95ee6f6b4cf272800f3dd5111c8eb27f887c5a38eda7bf29322d5806254a02 not found: ID does not exist"
Jul 18 00:47:13.933224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.933185    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"]
Jul 18 00:47:13.945901 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:13.945869    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9nrdt"]
Jul 18 00:47:14.900080 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:14.900039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt" event={"ID":"0972432b-1e47-4eea-9ba5-5d0ab0d94b55","Type":"ContainerStarted","Data":"945e1d112aad80b95c286febfab511481290d2a34e56f2ecf5a1c54d880c3d93"}
Jul 18 00:47:14.900548 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:14.900166    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:14.916794 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:14.916698    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt" podStartSLOduration=1.893549328 podStartE2EDuration="3.916678616s" totalImagesPullingTime="2.023129288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:14.915812693 +0000 UTC m=+414.443317647" watchObservedRunningTime="2026-07-18 00:47:14.916678616 +0000 UTC m=+414.444183580"
Jul 18 00:47:14.920141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:14.920112    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511ee99a-b253-42a7-8fdb-bedc95ec369c" path="/var/lib/kubelet/pods/511ee99a-b253-42a7-8fdb-bedc95ec369c/volumes"
Jul 18 00:47:15.850715 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:15.850682    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lkhmr"
Jul 18 00:47:16.852822 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:16.852788    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gpxfc"
Jul 18 00:47:45.911958 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:45.911877    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8dkt"
Jul 18 00:47:59.838797 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.838735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:59.934643 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.934609    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:47:59.949332 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.949303    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnpml"]
Jul 18 00:47:59.949645 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.949632    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511ee99a-b253-42a7-8fdb-bedc95ec369c" containerName="discovery"
Jul 18 00:47:59.949690 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.949647    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="511ee99a-b253-42a7-8fdb-bedc95ec369c" containerName="discovery"
Jul 18 00:47:59.949722 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.949699    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="511ee99a-b253-42a7-8fdb-bedc95ec369c" containerName="discovery"
Jul 18 00:47:59.952730 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.952710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:47:59.954992 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.954969    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5c6hb\""
Jul 18 00:47:59.955117 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.955000    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 00:47:59.955117 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.954991    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 00:47:59.962202 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:47:59.962173    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnpml"]
Jul 18 00:48:00.001377 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.001334    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:00.031582 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.031551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh6ps\" (UniqueName: \"kubernetes.io/projected/6194320f-4659-4521-b36d-f7f9e199e0fd-kube-api-access-rh6ps\") pod \"kuadrant-operator-catalog-jnpml\" (UID: \"6194320f-4659-4521-b36d-f7f9e199e0fd\") " pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:48:00.132834 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.132752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh6ps\" (UniqueName: \"kubernetes.io/projected/6194320f-4659-4521-b36d-f7f9e199e0fd-kube-api-access-rh6ps\") pod \"kuadrant-operator-catalog-jnpml\" (UID: \"6194320f-4659-4521-b36d-f7f9e199e0fd\") " pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:48:00.143873 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.143837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh6ps\" (UniqueName: \"kubernetes.io/projected/6194320f-4659-4521-b36d-f7f9e199e0fd-kube-api-access-rh6ps\") pod \"kuadrant-operator-catalog-jnpml\" (UID: \"6194320f-4659-4521-b36d-f7f9e199e0fd\") " pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:48:00.264326 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.264254    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:48:00.324638 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.324610    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnpml"]
Jul 18 00:48:00.402007 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.401977    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnpml"]
Jul 18 00:48:00.404941 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:48:00.404906    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6194320f_4659_4521_b36d_f7f9e199e0fd.slice/crio-25ffc29255a032a0b44f4c568e356d6b9d9b271a6006dedddc1f2de0e8832474 WatchSource:0}: Error finding container 25ffc29255a032a0b44f4c568e356d6b9d9b271a6006dedddc1f2de0e8832474: Status 404 returned error can't find the container with id 25ffc29255a032a0b44f4c568e356d6b9d9b271a6006dedddc1f2de0e8832474
Jul 18 00:48:00.531948 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.531912    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm4kd"]
Jul 18 00:48:00.536513 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.536494    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:00.544549 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.544529    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm4kd"]
Jul 18 00:48:00.637613 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.637576    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wf7\" (UniqueName: \"kubernetes.io/projected/cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d-kube-api-access-79wf7\") pod \"kuadrant-operator-catalog-pm4kd\" (UID: \"cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d\") " pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:00.738154 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.738063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wf7\" (UniqueName: \"kubernetes.io/projected/cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d-kube-api-access-79wf7\") pod \"kuadrant-operator-catalog-pm4kd\" (UID: \"cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d\") " pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:00.746124 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.746102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wf7\" (UniqueName: \"kubernetes.io/projected/cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d-kube-api-access-79wf7\") pod \"kuadrant-operator-catalog-pm4kd\" (UID: \"cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d\") " pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:00.747690 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.747660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:00.847113 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.847081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:00.958040 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:00.957992    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:01.053698 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:01.053669    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm4kd"]
Jul 18 00:48:01.076806 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:48:01.076761    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf441afe_bfe3_4e0d_a8ef_2b41f7c02e4d.slice/crio-39891c8bcfac9b47b7800d6bbfa8ae2225240fe54f8ba62fa2a316ddecd6dc19 WatchSource:0}: Error finding container 39891c8bcfac9b47b7800d6bbfa8ae2225240fe54f8ba62fa2a316ddecd6dc19: Status 404 returned error can't find the container with id 39891c8bcfac9b47b7800d6bbfa8ae2225240fe54f8ba62fa2a316ddecd6dc19
Jul 18 00:48:01.105868 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:01.105833    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnpml" event={"ID":"6194320f-4659-4521-b36d-f7f9e199e0fd","Type":"ContainerStarted","Data":"25ffc29255a032a0b44f4c568e356d6b9d9b271a6006dedddc1f2de0e8832474"}
Jul 18 00:48:01.107027 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:01.106999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd" event={"ID":"cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d","Type":"ContainerStarted","Data":"39891c8bcfac9b47b7800d6bbfa8ae2225240fe54f8ba62fa2a316ddecd6dc19"}
Jul 18 00:48:03.119627 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.119590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd" event={"ID":"cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d","Type":"ContainerStarted","Data":"5ce28b05c69aa7a033f91cc889bcd98bdd396442ed4dc92f0ed99b27e1db46fa"}
Jul 18 00:48:03.120986 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.120961    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnpml" event={"ID":"6194320f-4659-4521-b36d-f7f9e199e0fd","Type":"ContainerStarted","Data":"88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03"}
Jul 18 00:48:03.121045 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.120986    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jnpml" podUID="6194320f-4659-4521-b36d-f7f9e199e0fd" containerName="registry-server" containerID="cri-o://88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03" gracePeriod=2
Jul 18 00:48:03.134377 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.134322    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd" podStartSLOduration=1.810523479 podStartE2EDuration="3.134306666s" totalImagesPullingTime="1.323783187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:48:03.133412768 +0000 UTC m=+462.660917735" watchObservedRunningTime="2026-07-18 00:48:03.134306666 +0000 UTC m=+462.661811627"
Jul 18 00:48:03.148732 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.148682    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jnpml" podStartSLOduration=2.158223056 podStartE2EDuration="4.148664727s" totalImagesPullingTime="1.990441671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:48:03.147095571 +0000 UTC m=+462.674600530" watchObservedRunningTime="2026-07-18 00:48:03.148664727 +0000 UTC m=+462.676169687"
Jul 18 00:48:03.368871 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.368844    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:48:03.463568 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.463477    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh6ps\" (UniqueName: \"kubernetes.io/projected/6194320f-4659-4521-b36d-f7f9e199e0fd-kube-api-access-rh6ps\") pod \"6194320f-4659-4521-b36d-f7f9e199e0fd\" (UID: \"6194320f-4659-4521-b36d-f7f9e199e0fd\") "
Jul 18 00:48:03.465976 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.465936    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6194320f-4659-4521-b36d-f7f9e199e0fd-kube-api-access-rh6ps" (OuterVolumeSpecName: "kube-api-access-rh6ps") pod "6194320f-4659-4521-b36d-f7f9e199e0fd" (UID: "6194320f-4659-4521-b36d-f7f9e199e0fd"). InnerVolumeSpecName "kube-api-access-rh6ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:48:03.565072 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:03.565033    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh6ps\" (UniqueName: \"kubernetes.io/projected/6194320f-4659-4521-b36d-f7f9e199e0fd-kube-api-access-rh6ps\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:48:04.126069 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.125977    2567 generic.go:356] "Generic (PLEG): container finished" podID="6194320f-4659-4521-b36d-f7f9e199e0fd" containerID="88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03" exitCode=0
Jul 18 00:48:04.126069 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.126023    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnpml"
Jul 18 00:48:04.126598 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.126065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnpml" event={"ID":"6194320f-4659-4521-b36d-f7f9e199e0fd","Type":"ContainerDied","Data":"88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03"}
Jul 18 00:48:04.126598 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.126114    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnpml" event={"ID":"6194320f-4659-4521-b36d-f7f9e199e0fd","Type":"ContainerDied","Data":"25ffc29255a032a0b44f4c568e356d6b9d9b271a6006dedddc1f2de0e8832474"}
Jul 18 00:48:04.126598 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.126134    2567 scope.go:117] "RemoveContainer" containerID="88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03"
Jul 18 00:48:04.135241 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.135221    2567 scope.go:117] "RemoveContainer" containerID="88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03"
Jul 18 00:48:04.135526 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:04.135510    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03\": container with ID starting with 88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03 not found: ID does not exist" containerID="88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03"
Jul 18 00:48:04.135591 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.135535    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03"} err="failed to get container status \"88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03\": rpc error: code = NotFound desc = could not find container \"88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03\": container with ID starting with 88195059c5605b9638c834703653f626fcf36512db6841744abbe12e26b8dd03 not found: ID does not exist"
Jul 18 00:48:04.145475 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.145453    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnpml"]
Jul 18 00:48:04.147311 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.147292    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnpml"]
Jul 18 00:48:04.918223 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:04.918192    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6194320f-4659-4521-b36d-f7f9e199e0fd" path="/var/lib/kubelet/pods/6194320f-4659-4521-b36d-f7f9e199e0fd/volumes"
Jul 18 00:48:10.847721 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:10.847677    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:10.848181 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:10.847762    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:10.871788 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:10.871759    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:11.184249 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:11.184179    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pm4kd"
Jul 18 00:48:15.730186 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:15.730155    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:26.892063 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:26.892023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:29.927418 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.927384    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"]
Jul 18 00:48:29.927788 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.927694    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6194320f-4659-4521-b36d-f7f9e199e0fd" containerName="registry-server"
Jul 18 00:48:29.927788 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.927704    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6194320f-4659-4521-b36d-f7f9e199e0fd" containerName="registry-server"
Jul 18 00:48:29.927857 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.927800    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6194320f-4659-4521-b36d-f7f9e199e0fd" containerName="registry-server"
Jul 18 00:48:29.932110 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.932089    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:29.934557 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.934536    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nwffl\""
Jul 18 00:48:29.942640 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.942619    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"]
Jul 18 00:48:29.981434 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:29.981396    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6l5l\" (UniqueName: \"kubernetes.io/projected/06a0236c-b644-41d9-be22-c01531b661f7-kube-api-access-v6l5l\") pod \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" (UID: \"06a0236c-b644-41d9-be22-c01531b661f7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:30.082675 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:30.082637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6l5l\" (UniqueName: \"kubernetes.io/projected/06a0236c-b644-41d9-be22-c01531b661f7-kube-api-access-v6l5l\") pod \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" (UID: \"06a0236c-b644-41d9-be22-c01531b661f7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:30.097616 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:30.097580    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6l5l\" (UniqueName: \"kubernetes.io/projected/06a0236c-b644-41d9-be22-c01531b661f7-kube-api-access-v6l5l\") pod \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" (UID: \"06a0236c-b644-41d9-be22-c01531b661f7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:30.242978 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:30.242885    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:30.381211 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:30.381184    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"]
Jul 18 00:48:30.383998 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:48:30.383955    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a0236c_b644_41d9_be22_c01531b661f7.slice/crio-dece1f743431c6cde84560f1455947e330e547f0d08cd91de5a60ac312fbc1c7 WatchSource:0}: Error finding container dece1f743431c6cde84560f1455947e330e547f0d08cd91de5a60ac312fbc1c7: Status 404 returned error can't find the container with id dece1f743431c6cde84560f1455947e330e547f0d08cd91de5a60ac312fbc1c7
Jul 18 00:48:31.260338 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:31.260298    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x" event={"ID":"06a0236c-b644-41d9-be22-c01531b661f7","Type":"ContainerStarted","Data":"dece1f743431c6cde84560f1455947e330e547f0d08cd91de5a60ac312fbc1c7"}
Jul 18 00:48:31.564666 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:31.564614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:32.265973 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:32.265935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x" event={"ID":"06a0236c-b644-41d9-be22-c01531b661f7","Type":"ContainerStarted","Data":"9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212"}
Jul 18 00:48:32.266401 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:32.266001    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:32.285973 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:32.285911    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x" podStartSLOduration=2.05736923 podStartE2EDuration="3.285895078s" totalImagesPullingTime="1.228525848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:48:32.284862448 +0000 UTC m=+491.812367408" watchObservedRunningTime="2026-07-18 00:48:32.285895078 +0000 UTC m=+491.813400039"
Jul 18 00:48:33.605398 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.605365    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"]
Jul 18 00:48:33.608823 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.608798    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:33.611312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.611291    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 18 00:48:33.611431 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.611301    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sb9qc\""
Jul 18 00:48:33.624119 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.624094    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"]
Jul 18 00:48:33.715525 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.715481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-757g4\" (UniqueName: \"kubernetes.io/projected/3941225a-1acf-4693-8031-e26fd4904303-kube-api-access-757g4\") pod \"dns-operator-controller-manager-864dd78c4c-4qgrv\" (UID: \"3941225a-1acf-4693-8031-e26fd4904303\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:33.816604 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.816571    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-757g4\" (UniqueName: \"kubernetes.io/projected/3941225a-1acf-4693-8031-e26fd4904303-kube-api-access-757g4\") pod \"dns-operator-controller-manager-864dd78c4c-4qgrv\" (UID: \"3941225a-1acf-4693-8031-e26fd4904303\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:33.827197 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.827166    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-757g4\" (UniqueName: \"kubernetes.io/projected/3941225a-1acf-4693-8031-e26fd4904303-kube-api-access-757g4\") pod \"dns-operator-controller-manager-864dd78c4c-4qgrv\" (UID: \"3941225a-1acf-4693-8031-e26fd4904303\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:33.918733 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:33.918646    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:34.068451 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:34.068396    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"]
Jul 18 00:48:34.071287 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:48:34.071242    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3941225a_1acf_4693_8031_e26fd4904303.slice/crio-ec790c3c6f3c928d4faed11fe180de559c85d72ca71778aad7d6bc2a2ca109d6 WatchSource:0}: Error finding container ec790c3c6f3c928d4faed11fe180de559c85d72ca71778aad7d6bc2a2ca109d6: Status 404 returned error can't find the container with id ec790c3c6f3c928d4faed11fe180de559c85d72ca71778aad7d6bc2a2ca109d6
Jul 18 00:48:34.276155 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:34.276069    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv" event={"ID":"3941225a-1acf-4693-8031-e26fd4904303","Type":"ContainerStarted","Data":"ec790c3c6f3c928d4faed11fe180de559c85d72ca71778aad7d6bc2a2ca109d6"}
Jul 18 00:48:34.296483 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:34.296448    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:36.287326 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:36.287292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv" event={"ID":"3941225a-1acf-4693-8031-e26fd4904303","Type":"ContainerStarted","Data":"b984c603259042e3718171de9437d7feab382d5345f9a0db71e1a0eb2d528ea3"}
Jul 18 00:48:36.287716 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:36.287406    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:36.314487 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:36.314434    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv" podStartSLOduration=1.777218242 podStartE2EDuration="3.314416364s" totalImagesPullingTime="1.537198122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:48:36.312152008 +0000 UTC m=+495.839656970" watchObservedRunningTime="2026-07-18 00:48:36.314416364 +0000 UTC m=+495.841921323"
Jul 18 00:48:37.336155 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:37.336119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:38.455887 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.455852    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"]
Jul 18 00:48:38.462259 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.462235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.466101 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.466079    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hnt59\""
Jul 18 00:48:38.489808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.489774    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"]
Jul 18 00:48:38.560764 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.560725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwrvk\" (UniqueName: \"kubernetes.io/projected/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-kube-api-access-wwrvk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.560950 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.560826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.662003 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.661961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.662178 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.662033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrvk\" (UniqueName: \"kubernetes.io/projected/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-kube-api-access-wwrvk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.662433 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.662407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.673234 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.673212    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwrvk\" (UniqueName: \"kubernetes.io/projected/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-kube-api-access-wwrvk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.771844 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.771754    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:38.776843 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.776816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:38.922857 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.922832    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"]
Jul 18 00:48:38.923884 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:48:38.923859    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556ab50e_aeb1_46fe_ba94_e5ca3a98c8bd.slice/crio-50c1d97af06ae8b68a4a2bb9dd2359834ab24e416b862b5f916e32c918aefca4 WatchSource:0}: Error finding container 50c1d97af06ae8b68a4a2bb9dd2359834ab24e416b862b5f916e32c918aefca4: Status 404 returned error can't find the container with id 50c1d97af06ae8b68a4a2bb9dd2359834ab24e416b862b5f916e32c918aefca4
Jul 18 00:48:38.933874 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:38.933849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:39.302893 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:39.302860    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7" event={"ID":"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd","Type":"ContainerStarted","Data":"50c1d97af06ae8b68a4a2bb9dd2359834ab24e416b862b5f916e32c918aefca4"}
Jul 18 00:48:39.497830 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:39.497802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:40.568484 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:40.568431    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:41.896601 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:41.896564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:41.954357 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:41.954303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:43.186695 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:43.186663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:43.273313 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:43.273284    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:43.326515 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:43.326486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7" event={"ID":"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd","Type":"ContainerStarted","Data":"c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0"}
Jul 18 00:48:43.326683 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:43.326667    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:43.353978 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:43.353904    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7" podStartSLOduration=1.7048640750000001 podStartE2EDuration="5.353882015s" totalImagesPullingTime="3.64901794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:48:43.35016808 +0000 UTC m=+502.877673041" watchObservedRunningTime="2026-07-18 00:48:43.353882015 +0000 UTC m=+502.881386971"
Jul 18 00:48:44.576498 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:44.576465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:44.712613 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:44.712579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:45.344861 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.344826    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"]
Jul 18 00:48:45.347991 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.347966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:45.348668 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.348654    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.350988 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.350960    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 18 00:48:45.351079 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.350988    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xm56t\""
Jul 18 00:48:45.351079 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.351033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 18 00:48:45.357121 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.357099    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"]
Jul 18 00:48:45.423579 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.423528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c54113c7-77bf-4685-b1bb-06b267b7c8cb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.423772 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.423676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c54113c7-77bf-4685-b1bb-06b267b7c8cb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.423772 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.423743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbfj5\" (UniqueName: \"kubernetes.io/projected/c54113c7-77bf-4685-b1bb-06b267b7c8cb-kube-api-access-rbfj5\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.524758 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.524633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c54113c7-77bf-4685-b1bb-06b267b7c8cb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.524758 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.524714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c54113c7-77bf-4685-b1bb-06b267b7c8cb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.524758 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.524752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbfj5\" (UniqueName: \"kubernetes.io/projected/c54113c7-77bf-4685-b1bb-06b267b7c8cb-kube-api-access-rbfj5\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.525653 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.525626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c54113c7-77bf-4685-b1bb-06b267b7c8cb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.527647 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.527616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c54113c7-77bf-4685-b1bb-06b267b7c8cb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.532911 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.532882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbfj5\" (UniqueName: \"kubernetes.io/projected/c54113c7-77bf-4685-b1bb-06b267b7c8cb-kube-api-access-rbfj5\") pod \"kuadrant-console-plugin-66b5774985-fslf6\" (UID: \"c54113c7-77bf-4685-b1bb-06b267b7c8cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.564818 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.564771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:45.658387 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.658300    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"
Jul 18 00:48:45.797819 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:45.797787    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6"]
Jul 18 00:48:45.800541 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:48:45.800510    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc54113c7_77bf_4685_b1bb_06b267b7c8cb.slice/crio-037c8966740b57875e9dc1dae81f5bef3e5d5889fae27f64888f8690b17742e9 WatchSource:0}: Error finding container 037c8966740b57875e9dc1dae81f5bef3e5d5889fae27f64888f8690b17742e9: Status 404 returned error can't find the container with id 037c8966740b57875e9dc1dae81f5bef3e5d5889fae27f64888f8690b17742e9
Jul 18 00:48:46.345073 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:46.345030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6" event={"ID":"c54113c7-77bf-4685-b1bb-06b267b7c8cb","Type":"ContainerStarted","Data":"037c8966740b57875e9dc1dae81f5bef3e5d5889fae27f64888f8690b17742e9"}
Jul 18 00:48:46.511438 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:46.511400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:46.611364 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:46.611279    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:47.295887 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:47.295853    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4qgrv"
Jul 18 00:48:51.952368 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:51.952320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:52.080397 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:52.080356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:52.604870 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:52.604831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:52.735257 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:52.735220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:52.838141 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:52.838113    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:48:54.334146 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:54.334115    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:55.985102 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:55.985067    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"]
Jul 18 00:48:55.985588 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:55.985284    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" containerName="manager" containerID="cri-o://c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0" gracePeriod=2
Jul 18 00:48:55.991944 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:55.991912    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"]
Jul 18 00:48:55.998203 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:55.998153    2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\": pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:56.007357 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.007245    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"]
Jul 18 00:48:56.007987 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.007792    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x" podUID="06a0236c-b644-41d9-be22-c01531b661f7" containerName="manager" containerID="cri-o://9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212" gracePeriod=2
Jul 18 00:48:56.010208 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.010185    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"]
Jul 18 00:48:56.010819 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.010707    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" containerName="manager"
Jul 18 00:48:56.010819 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.010727    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" containerName="manager"
Jul 18 00:48:56.010988 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.010839    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" containerName="manager"
Jul 18 00:48:56.023547 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.023379    2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"06a0236c-b644-41d9-be22-c01531b661f7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-qnx2x\": pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:56.031925 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.026631    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:56.072597 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.072090    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"]
Jul 18 00:48:56.072597 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.072234    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.072807 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.072240    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"]
Jul 18 00:48:56.072807 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.072717    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"]
Jul 18 00:48:56.073330 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.073310    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06a0236c-b644-41d9-be22-c01531b661f7" containerName="manager"
Jul 18 00:48:56.073451 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.073334    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="06a0236c-b644-41d9-be22-c01531b661f7" containerName="manager"
Jul 18 00:48:56.073451 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.073421    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="06a0236c-b644-41d9-be22-c01531b661f7" containerName="manager"
Jul 18 00:48:56.075099 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.074982    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:56.078191 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.078158    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"]
Jul 18 00:48:56.078338 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.078318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:48:56.097838 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.097798    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="06a0236c-b644-41d9-be22-c01531b661f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:56.099654 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.099617    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="06a0236c-b644-41d9-be22-c01531b661f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:56.101520 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.101494    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:56.225097 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.225057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzzgq\" (UniqueName: \"kubernetes.io/projected/cf8d77a7-8d93-4cc4-91bd-686536ed9158-kube-api-access-mzzgq\") pod \"limitador-operator-controller-manager-859d7b78bd-xpj7w\" (UID: \"cf8d77a7-8d93-4cc4-91bd-686536ed9158\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:48:56.225320 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.225191    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87b54cb-187b-43a8-b605-d56eb1e550a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crfj5\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.225320 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.225218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c455p\" (UniqueName: \"kubernetes.io/projected/a87b54cb-187b-43a8-b605-d56eb1e550a2-kube-api-access-c455p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crfj5\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.243984 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.243905    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"]
Jul 18 00:48:56.254331 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.254302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.263554 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.263525    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"]
Jul 18 00:48:56.285414 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.285371    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:48:56.287463 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:48:56.287425    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="06a0236c-b644-41d9-be22-c01531b661f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:48:56.326689 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.326650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzzgq\" (UniqueName: \"kubernetes.io/projected/cf8d77a7-8d93-4cc4-91bd-686536ed9158-kube-api-access-mzzgq\") pod \"limitador-operator-controller-manager-859d7b78bd-xpj7w\" (UID: \"cf8d77a7-8d93-4cc4-91bd-686536ed9158\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:48:56.326689 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.326702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghv5t\" (UniqueName: \"kubernetes.io/projected/81723458-bf74-4b7d-ab11-6466e13bcd75-kube-api-access-ghv5t\") pod \"kuadrant-operator-controller-manager-86878c769f-6tx2c\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.326944 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.326790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87b54cb-187b-43a8-b605-d56eb1e550a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crfj5\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.326944 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.326813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c455p\" (UniqueName: \"kubernetes.io/projected/a87b54cb-187b-43a8-b605-d56eb1e550a2-kube-api-access-c455p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crfj5\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.326944 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.326845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81723458-bf74-4b7d-ab11-6466e13bcd75-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6tx2c\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.327186 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.327164    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87b54cb-187b-43a8-b605-d56eb1e550a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crfj5\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.339857 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.339825    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzzgq\" (UniqueName: \"kubernetes.io/projected/cf8d77a7-8d93-4cc4-91bd-686536ed9158-kube-api-access-mzzgq\") pod \"limitador-operator-controller-manager-859d7b78bd-xpj7w\" (UID: \"cf8d77a7-8d93-4cc4-91bd-686536ed9158\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:48:56.340032 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.339840    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c455p\" (UniqueName: \"kubernetes.io/projected/a87b54cb-187b-43a8-b605-d56eb1e550a2-kube-api-access-c455p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-crfj5\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.427459 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.427423    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81723458-bf74-4b7d-ab11-6466e13bcd75-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6tx2c\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.427639 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.427494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghv5t\" (UniqueName: \"kubernetes.io/projected/81723458-bf74-4b7d-ab11-6466e13bcd75-kube-api-access-ghv5t\") pod \"kuadrant-operator-controller-manager-86878c769f-6tx2c\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.427910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.427883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81723458-bf74-4b7d-ab11-6466e13bcd75-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6tx2c\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.437752 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.437688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghv5t\" (UniqueName: \"kubernetes.io/projected/81723458-bf74-4b7d-ab11-6466e13bcd75-kube-api-access-ghv5t\") pod \"kuadrant-operator-controller-manager-86878c769f-6tx2c\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:56.485406 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.485366    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:48:56.491537 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.491498    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:48:56.567222 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:56.567182    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:48:59.954249 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:48:59.954215    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:00.053759 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:00.053725    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:02.525411 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:02.525375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:02.626465 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:02.626415    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:04.932640 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:04.932582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:05.033492 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:05.033452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:06.659351 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.659321    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:49:06.661428 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:06.661398    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="06a0236c-b644-41d9-be22-c01531b661f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:49:06.683640 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.683606    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:49:06.685915 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:06.685880    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="06a0236c-b644-41d9-be22-c01531b661f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:49:06.687942 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:06.687906    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:49:06.708614 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.708590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"]
Jul 18 00:49:06.710835 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:49:06.710812    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81723458_bf74_4b7d_ab11_6466e13bcd75.slice/crio-8e1fb2824b3abd75b642cfb059a354beca2f32f2ae9b09f7388e6621fcf0e0a1 WatchSource:0}: Error finding container 8e1fb2824b3abd75b642cfb059a354beca2f32f2ae9b09f7388e6621fcf0e0a1: Status 404 returned error can't find the container with id 8e1fb2824b3abd75b642cfb059a354beca2f32f2ae9b09f7388e6621fcf0e0a1
Jul 18 00:49:06.831912 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.831874    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6l5l\" (UniqueName: \"kubernetes.io/projected/06a0236c-b644-41d9-be22-c01531b661f7-kube-api-access-v6l5l\") pod \"06a0236c-b644-41d9-be22-c01531b661f7\" (UID: \"06a0236c-b644-41d9-be22-c01531b661f7\") "
Jul 18 00:49:06.831912 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.831919    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-extensions-socket-volume\") pod \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") "
Jul 18 00:49:06.832114 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.832025    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwrvk\" (UniqueName: \"kubernetes.io/projected/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-kube-api-access-wwrvk\") pod \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\" (UID: \"556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd\") "
Jul 18 00:49:06.832446 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.832421    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" (UID: "556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 00:49:06.834166 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.834145    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06a0236c-b644-41d9-be22-c01531b661f7-kube-api-access-v6l5l" (OuterVolumeSpecName: "kube-api-access-v6l5l") pod "06a0236c-b644-41d9-be22-c01531b661f7" (UID: "06a0236c-b644-41d9-be22-c01531b661f7"). InnerVolumeSpecName "kube-api-access-v6l5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:49:06.834257 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.834172    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-kube-api-access-wwrvk" (OuterVolumeSpecName: "kube-api-access-wwrvk") pod "556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" (UID: "556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd"). InnerVolumeSpecName "kube-api-access-wwrvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:49:06.862618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.862592    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"]
Jul 18 00:49:06.865638 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.865590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"]
Jul 18 00:49:06.865834 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:49:06.865654    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8d77a7_8d93_4cc4_91bd_686536ed9158.slice/crio-d1fe2254b7f26e0c2c0b16070e21702038af4c5010d92ca75f11b111b7f21f02 WatchSource:0}: Error finding container d1fe2254b7f26e0c2c0b16070e21702038af4c5010d92ca75f11b111b7f21f02: Status 404 returned error can't find the container with id d1fe2254b7f26e0c2c0b16070e21702038af4c5010d92ca75f11b111b7f21f02
Jul 18 00:49:06.868438 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:49:06.868369    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87b54cb_187b_43a8_b605_d56eb1e550a2.slice/crio-08ecfbb5eb34dc067bea6a868edf78dd346d94629cb59661af3dedbb65384463 WatchSource:0}: Error finding container 08ecfbb5eb34dc067bea6a868edf78dd346d94629cb59661af3dedbb65384463: Status 404 returned error can't find the container with id 08ecfbb5eb34dc067bea6a868edf78dd346d94629cb59661af3dedbb65384463
Jul 18 00:49:06.919660 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.919633    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06a0236c-b644-41d9-be22-c01531b661f7" path="/var/lib/kubelet/pods/06a0236c-b644-41d9-be22-c01531b661f7/volumes"
Jul 18 00:49:06.919985 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.919973    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" path="/var/lib/kubelet/pods/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd/volumes"
Jul 18 00:49:06.932982 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.932951    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6l5l\" (UniqueName: \"kubernetes.io/projected/06a0236c-b644-41d9-be22-c01531b661f7-kube-api-access-v6l5l\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:49:06.932982 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.932979    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-extensions-socket-volume\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:49:06.933112 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:06.932995    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwrvk\" (UniqueName: \"kubernetes.io/projected/556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd-kube-api-access-wwrvk\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:49:07.469541 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.469507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5" event={"ID":"a87b54cb-187b-43a8-b605-d56eb1e550a2","Type":"ContainerStarted","Data":"8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8"}
Jul 18 00:49:07.469667 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.469564    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5" event={"ID":"a87b54cb-187b-43a8-b605-d56eb1e550a2","Type":"ContainerStarted","Data":"08ecfbb5eb34dc067bea6a868edf78dd346d94629cb59661af3dedbb65384463"}
Jul 18 00:49:07.469740 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.469674    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:49:07.471369 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.471335    2567 generic.go:356] "Generic (PLEG): container finished" podID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" containerID="c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0" exitCode=0
Jul 18 00:49:07.471503 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.471376    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:49:07.471503 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.471452    2567 scope.go:117] "RemoveContainer" containerID="c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0"
Jul 18 00:49:07.473367 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.473327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w" event={"ID":"cf8d77a7-8d93-4cc4-91bd-686536ed9158","Type":"ContainerStarted","Data":"1beaaec189a4a81416be0b4424b34e4f34aad8d10860fa8c53ac7d3ed0189280"}
Jul 18 00:49:07.473473 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.473386    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w" event={"ID":"cf8d77a7-8d93-4cc4-91bd-686536ed9158","Type":"ContainerStarted","Data":"d1fe2254b7f26e0c2c0b16070e21702038af4c5010d92ca75f11b111b7f21f02"}
Jul 18 00:49:07.473473 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.473440    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:49:07.475202 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.475172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c" event={"ID":"81723458-bf74-4b7d-ab11-6466e13bcd75","Type":"ContainerStarted","Data":"1bbd2c7695a5e4e50205370c1d360a4791618a766626489ece3b480a0aa09c28"}
Jul 18 00:49:07.475363 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.475226    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c" event={"ID":"81723458-bf74-4b7d-ab11-6466e13bcd75","Type":"ContainerStarted","Data":"8e1fb2824b3abd75b642cfb059a354beca2f32f2ae9b09f7388e6621fcf0e0a1"}
Jul 18 00:49:07.475363 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.475255    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:49:07.476966 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.476939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6" event={"ID":"c54113c7-77bf-4685-b1bb-06b267b7c8cb","Type":"ContainerStarted","Data":"697df9756187e300cf6ab90ce751c0c450e10667288e882ae79a553b92386d1b"}
Jul 18 00:49:07.478386 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.478364    2567 generic.go:356] "Generic (PLEG): container finished" podID="06a0236c-b644-41d9-be22-c01531b661f7" containerID="9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212" exitCode=0
Jul 18 00:49:07.478469 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.478395    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:49:07.481976 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.481961    2567 scope.go:117] "RemoveContainer" containerID="c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0"
Jul 18 00:49:07.482239 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:07.482221    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0\": container with ID starting with c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0 not found: ID does not exist" containerID="c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0"
Jul 18 00:49:07.482313 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.482248    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0"} err="failed to get container status \"c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0\": rpc error: code = NotFound desc = could not find container \"c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0\": container with ID starting with c60a7b03922884e88b5082e4bdce151393cff6093f7c9df1b5ff0a9b422d04d0 not found: ID does not exist"
Jul 18 00:49:07.482313 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.482296    2567 scope.go:117] "RemoveContainer" containerID="9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212"
Jul 18 00:49:07.489972 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.489932    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5" podStartSLOduration=12.489917268 podStartE2EDuration="12.489917268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:07.488655796 +0000 UTC m=+527.016160756" watchObservedRunningTime="2026-07-18 00:49:07.489917268 +0000 UTC m=+527.017422227"
Jul 18 00:49:07.490874 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.490774    2567 scope.go:117] "RemoveContainer" containerID="9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212"
Jul 18 00:49:07.491099 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:07.491075    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212\": container with ID starting with 9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212 not found: ID does not exist" containerID="9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212"
Jul 18 00:49:07.491158 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.491110    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212"} err="failed to get container status \"9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212\": rpc error: code = NotFound desc = could not find container \"9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212\": container with ID starting with 9ee84f521495e6936526b9c94a1fef20e6bd3c837d448ae3ef212def6d10c212 not found: ID does not exist"
Jul 18 00:49:07.505001 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.504955    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fslf6" podStartSLOduration=1.738171614 podStartE2EDuration="22.504943767s" totalImagesPullingTime="20.766772153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:07.503495829 +0000 UTC m=+527.031000810" watchObservedRunningTime="2026-07-18 00:49:07.504943767 +0000 UTC m=+527.032448727"
Jul 18 00:49:07.524504 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.524446    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c" podStartSLOduration=11.524425941 podStartE2EDuration="11.524425941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:07.522965767 +0000 UTC m=+527.050470729" watchObservedRunningTime="2026-07-18 00:49:07.524425941 +0000 UTC m=+527.051930901"
Jul 18 00:49:07.524897 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:07.524864    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng2l7\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="556ab50e-aeb1-46fe-ba94-e5ca3a98c8bd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng2l7"
Jul 18 00:49:07.548782 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:07.548723    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w" podStartSLOduration=11.548703016 podStartE2EDuration="11.548703016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:07.547046488 +0000 UTC m=+527.074551461" watchObservedRunningTime="2026-07-18 00:49:07.548703016 +0000 UTC m=+527.076207978"
Jul 18 00:49:07.549142 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:07.549116    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnx2x\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="06a0236c-b644-41d9-be22-c01531b661f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnx2x"
Jul 18 00:49:08.923132 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:08.923099    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:09.026325 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:09.026261    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:10.810729 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:10.810691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:10.878875 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:10.878845    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:12.669136 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:12.669105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:12.771405 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:12.771375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:17.762583 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:17.762537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:17.836964 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:17.836927    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:18.487554 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.487516    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:49:18.488551 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.488525    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 00:49:18.488683 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.488587    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpj7w"
Jul 18 00:49:18.560370 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.560335    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"]
Jul 18 00:49:18.560544 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.560526    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5" podUID="a87b54cb-187b-43a8-b605-d56eb1e550a2" containerName="manager" containerID="cri-o://8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8" gracePeriod=10
Jul 18 00:49:18.813616 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.813591    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:49:18.841838 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.841808    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c455p\" (UniqueName: \"kubernetes.io/projected/a87b54cb-187b-43a8-b605-d56eb1e550a2-kube-api-access-c455p\") pod \"a87b54cb-187b-43a8-b605-d56eb1e550a2\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") "
Jul 18 00:49:18.841838 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.841858    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87b54cb-187b-43a8-b605-d56eb1e550a2-extensions-socket-volume\") pod \"a87b54cb-187b-43a8-b605-d56eb1e550a2\" (UID: \"a87b54cb-187b-43a8-b605-d56eb1e550a2\") "
Jul 18 00:49:18.842283 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.842244    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a87b54cb-187b-43a8-b605-d56eb1e550a2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a87b54cb-187b-43a8-b605-d56eb1e550a2" (UID: "a87b54cb-187b-43a8-b605-d56eb1e550a2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 00:49:18.844354 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.844312    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a87b54cb-187b-43a8-b605-d56eb1e550a2-kube-api-access-c455p" (OuterVolumeSpecName: "kube-api-access-c455p") pod "a87b54cb-187b-43a8-b605-d56eb1e550a2" (UID: "a87b54cb-187b-43a8-b605-d56eb1e550a2"). InnerVolumeSpecName "kube-api-access-c455p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 00:49:18.942588 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.942552    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a87b54cb-187b-43a8-b605-d56eb1e550a2-extensions-socket-volume\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:49:18.942588 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:18.942585    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c455p\" (UniqueName: \"kubernetes.io/projected/a87b54cb-187b-43a8-b605-d56eb1e550a2-kube-api-access-c455p\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 00:49:19.541012 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.540976    2567 generic.go:356] "Generic (PLEG): container finished" podID="a87b54cb-187b-43a8-b605-d56eb1e550a2" containerID="8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8" exitCode=0
Jul 18 00:49:19.541182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.541032    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"
Jul 18 00:49:19.541182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.541060    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5" event={"ID":"a87b54cb-187b-43a8-b605-d56eb1e550a2","Type":"ContainerDied","Data":"8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8"}
Jul 18 00:49:19.541182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.541141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5" event={"ID":"a87b54cb-187b-43a8-b605-d56eb1e550a2","Type":"ContainerDied","Data":"08ecfbb5eb34dc067bea6a868edf78dd346d94629cb59661af3dedbb65384463"}
Jul 18 00:49:19.541182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.541164    2567 scope.go:117] "RemoveContainer" containerID="8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8"
Jul 18 00:49:19.551847 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.551827    2567 scope.go:117] "RemoveContainer" containerID="8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8"
Jul 18 00:49:19.552145 ip-10-0-135-55 kubenswrapper[2567]: E0718 00:49:19.552125    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8\": container with ID starting with 8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8 not found: ID does not exist" containerID="8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8"
Jul 18 00:49:19.552188 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.552156    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8"} err="failed to get container status \"8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8\": rpc error: code = NotFound desc = could not find container \"8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8\": container with ID starting with 8911862c2e702d5bf935b117905bd44b103f8c472f3865a0a682b6e51a4a64e8 not found: ID does not exist"
Jul 18 00:49:19.565707 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.565674    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"]
Jul 18 00:49:19.569558 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:19.569532    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-crfj5"]
Jul 18 00:49:20.354808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:20.354775    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:20.463741 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:20.463708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:20.919822 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:20.919787    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87b54cb-187b-43a8-b605-d56eb1e550a2" path="/var/lib/kubelet/pods/a87b54cb-187b-43a8-b605-d56eb1e550a2/volumes"
Jul 18 00:49:23.360296 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:23.360236    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:23.460307 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:23.460250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:27.156673 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:27.156628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:27.257849 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:27.257815    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:34.828378 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.828337    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"]
Jul 18 00:49:34.828935 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.828906    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87b54cb-187b-43a8-b605-d56eb1e550a2" containerName="manager"
Jul 18 00:49:34.828935 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.828933    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87b54cb-187b-43a8-b605-d56eb1e550a2" containerName="manager"
Jul 18 00:49:34.829119 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.829033    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87b54cb-187b-43a8-b605-d56eb1e550a2" containerName="manager"
Jul 18 00:49:34.850163 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.850130    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:34.971893 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.971855    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"]
Jul 18 00:49:34.972142 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.971927    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:34.974371 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:34.974348    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-95mtm\""
Jul 18 00:49:35.085293 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57719e6a-9f81-47e9-a008-69cd60d41d2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085293 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085486 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085348    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085486 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085486 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqfc\" (UniqueName: \"kubernetes.io/projected/57719e6a-9f81-47e9-a008-69cd60d41d2c-kube-api-access-djqfc\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085486 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085639 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085639 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.085639 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.085573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186344 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186355    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186419    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57719e6a-9f81-47e9-a008-69cd60d41d2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186826 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186826 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqfc\" (UniqueName: \"kubernetes.io/projected/57719e6a-9f81-47e9-a008-69cd60d41d2c-kube-api-access-djqfc\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186926 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.186982 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.186945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.187044 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.187005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.187301 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.187253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.187779 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.187732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57719e6a-9f81-47e9-a008-69cd60d41d2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.189321 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.189301    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.189537 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.189520    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.194082 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.194055    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57719e6a-9f81-47e9-a008-69cd60d41d2c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.194318 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.194299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqfc\" (UniqueName: \"kubernetes.io/projected/57719e6a-9f81-47e9-a008-69cd60d41d2c-kube-api-access-djqfc\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgrtr\" (UID: \"57719e6a-9f81-47e9-a008-69cd60d41d2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.220169 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.220139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:35.283294 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.283246    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:35.457182 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.457160    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"]
Jul 18 00:49:35.459501 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:49:35.459474    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57719e6a_9f81_47e9_a008_69cd60d41d2c.slice/crio-8b853f1bc62738d093360e754768d49ba374ac0f9a26e5f7451f906324f52b21 WatchSource:0}: Error finding container 8b853f1bc62738d093360e754768d49ba374ac0f9a26e5f7451f906324f52b21: Status 404 returned error can't find the container with id 8b853f1bc62738d093360e754768d49ba374ac0f9a26e5f7451f906324f52b21
Jul 18 00:49:35.461592 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.461560    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:49:35.461652 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.461628    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:49:35.461693 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.461657    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:49:35.627415 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.627311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" event={"ID":"57719e6a-9f81-47e9-a008-69cd60d41d2c","Type":"ContainerStarted","Data":"1fb6d9a817d907478c41f417380abd096588970ba5f949626e990c22141cda5a"}
Jul 18 00:49:35.627415 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.627385    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" event={"ID":"57719e6a-9f81-47e9-a008-69cd60d41d2c","Type":"ContainerStarted","Data":"8b853f1bc62738d093360e754768d49ba374ac0f9a26e5f7451f906324f52b21"}
Jul 18 00:49:35.644968 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:35.644915    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" podStartSLOduration=1.644898472 podStartE2EDuration="1.644898472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:49:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:35.643660993 +0000 UTC m=+555.171165952" watchObservedRunningTime="2026-07-18 00:49:35.644898472 +0000 UTC m=+555.172403432"
Jul 18 00:49:36.283521 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.283491    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:36.288575 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.288544    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:36.371105 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.371075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:36.466998 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.466964    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:36.633640 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.633609    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:36.634743 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.634724    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:49:36.639312 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.639288    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:36.797319 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.797291    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:36.930828 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:36.930733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:37.042089 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.042050    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:49:37.084833 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.084789    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:49:37.084992 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.084844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:49:37.084992 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.084927    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.087016 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.086990    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 18 00:49:37.204326 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.204207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w79xp\" (UniqueName: \"kubernetes.io/projected/09578043-ec7d-4407-9345-b15116d06fcf-kube-api-access-w79xp\") pod \"limitador-limitador-54d6d988d5-52mmm\" (UID: \"09578043-ec7d-4407-9345-b15116d06fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.204468 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.204360    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/09578043-ec7d-4407-9345-b15116d06fcf-config-file\") pod \"limitador-limitador-54d6d988d5-52mmm\" (UID: \"09578043-ec7d-4407-9345-b15116d06fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.305662 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.305627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/09578043-ec7d-4407-9345-b15116d06fcf-config-file\") pod \"limitador-limitador-54d6d988d5-52mmm\" (UID: \"09578043-ec7d-4407-9345-b15116d06fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.306192 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.305681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w79xp\" (UniqueName: \"kubernetes.io/projected/09578043-ec7d-4407-9345-b15116d06fcf-kube-api-access-w79xp\") pod \"limitador-limitador-54d6d988d5-52mmm\" (UID: \"09578043-ec7d-4407-9345-b15116d06fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.306465 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.306440    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/09578043-ec7d-4407-9345-b15116d06fcf-config-file\") pod \"limitador-limitador-54d6d988d5-52mmm\" (UID: \"09578043-ec7d-4407-9345-b15116d06fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.314242 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.314217    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w79xp\" (UniqueName: \"kubernetes.io/projected/09578043-ec7d-4407-9345-b15116d06fcf-kube-api-access-w79xp\") pod \"limitador-limitador-54d6d988d5-52mmm\" (UID: \"09578043-ec7d-4407-9345-b15116d06fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.396808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.396758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:37.521927 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.521889    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:37.540749 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.540726    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:49:37.542684 ip-10-0-135-55 kubenswrapper[2567]: W0718 00:49:37.542659    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09578043_ec7d_4407_9345_b15116d06fcf.slice/crio-bb8847d7c8b7177a2abd4eb312a21dfed87393484bbe1afc089fabc3297c3ae0 WatchSource:0}: Error finding container bb8847d7c8b7177a2abd4eb312a21dfed87393484bbe1afc089fabc3297c3ae0: Status 404 returned error can't find the container with id bb8847d7c8b7177a2abd4eb312a21dfed87393484bbe1afc089fabc3297c3ae0
Jul 18 00:49:37.613308 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.613238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:37.639341 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:37.639309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm" event={"ID":"09578043-ec7d-4407-9345-b15116d06fcf","Type":"ContainerStarted","Data":"bb8847d7c8b7177a2abd4eb312a21dfed87393484bbe1afc089fabc3297c3ae0"}
Jul 18 00:49:40.658573 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:40.658528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm" event={"ID":"09578043-ec7d-4407-9345-b15116d06fcf","Type":"ContainerStarted","Data":"13a18bc14a6fbb9f12c395c0914692072f6df702bd5daa5fe5a6a93f7a43a97b"}
Jul 18 00:49:40.659013 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:40.658598    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:40.674923 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:40.674870    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm" podStartSLOduration=1.150185889 podStartE2EDuration="3.674855463s" totalImagesPullingTime="2.524669574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:40.673341109 +0000 UTC m=+560.200846069" watchObservedRunningTime="2026-07-18 00:49:40.674855463 +0000 UTC m=+560.202360423"
Jul 18 00:49:51.670042 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:51.670006    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-52mmm"
Jul 18 00:49:52.703326 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:52.703258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:52.800497 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:52.800466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:53.187287 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:53.187249    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:49:58.120410 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:49:58.120370    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:05.017579 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:05.017540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:06.133157 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:06.133124    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:11.995737 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:11.995705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:12.089323 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:12.089262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:12.167019 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:12.166984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:12.575708 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:12.575669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:12.923586 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:12.923497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:16.229328 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:16.229294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:16.346080 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:16.346044    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:20.854564 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:20.854531    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:50:20.858577 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:20.858553    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:50:47.934967 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:47.934870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:48.021641 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:48.021595    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:48.440548 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:48.440497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:48.581661 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:48.581632    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:49.019720 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:49.019688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:49.115831 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:49.115790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:49.217475 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:49.217445    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:50:57.033618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.033517    2567 generic.go:356] "Generic (PLEG): container finished" podID="57719e6a-9f81-47e9-a008-69cd60d41d2c" containerID="1fb6d9a817d907478c41f417380abd096588970ba5f949626e990c22141cda5a" exitCode=0
Jul 18 00:50:57.033618 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.033565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" event={"ID":"57719e6a-9f81-47e9-a008-69cd60d41d2c","Type":"ContainerDied","Data":"1fb6d9a817d907478c41f417380abd096588970ba5f949626e990c22141cda5a"}
Jul 18 00:50:57.034077 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.033921    2567 scope.go:117] "RemoveContainer" containerID="1fb6d9a817d907478c41f417380abd096588970ba5f949626e990c22141cda5a"
Jul 18 00:50:57.034952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.034917    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:50:57.035088 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.035007    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:50:57.035153 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.035110    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 00:50:57.974152 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:57.974102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:58.040547 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:58.040506    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" event={"ID":"57719e6a-9f81-47e9-a008-69cd60d41d2c","Type":"ContainerStarted","Data":"cffaf8d2aed0deb756d6951976ae80e8a02bf3a5844f92ba17340c29b09f9d03"}
Jul 18 00:50:58.066285 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:58.066221    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:50:58.283532 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:58.283437    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:50:58.283685 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:58.283662    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgrtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 18 00:50:58.283868 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:58.283720    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" podUID="57719e6a-9f81-47e9-a008-69cd60d41d2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 18 00:50:59.283824 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:59.283783    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgrtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 18 00:50:59.284379 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:59.283866    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" podUID="57719e6a-9f81-47e9-a008-69cd60d41d2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 18 00:50:59.420511 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:50:59.420471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:00.284110 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:00.284070    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgrtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 18 00:51:00.284533 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:00.284133    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" podUID="57719e6a-9f81-47e9-a008-69cd60d41d2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 18 00:51:01.284697 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:01.284660    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgrtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 18 00:51:01.285101 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:01.284729    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr" podUID="57719e6a-9f81-47e9-a008-69cd60d41d2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 18 00:51:01.851415 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:01.851374    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:02.288253 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:02.288170    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:51:02.288639 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:02.288621    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:51:02.289512 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:02.289489    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgrtr"
Jul 18 00:51:02.407808 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:02.407775    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:51:10.875525 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:10.875493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:22.371536 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:22.371497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:23.235136 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:23.235103    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:51:46.606762 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:46.606717    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:46.708499 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:46.708463    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:46.955453 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:46.955367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:48.208593 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:48.208562    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:51:57.313015 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:57.312966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:51:58.310183 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:51:58.310148    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:52:07.724796 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:52:07.724763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:52:08.991786 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:52:08.991752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:52:09.818918 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:52:09.818884    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:52:20.126335 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:52:20.126234    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:52:22.610841 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:52:22.610800    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:53:35.571425 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:53:35.571383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:53:35.660181 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:53:35.660151    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:53:36.649417 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:53:36.649381    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:53:36.746812 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:53:36.746777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:54:03.109392 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:54:03.109295    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:54:14.700306 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:54:14.699498    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:54:24.510869 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:54:24.510820    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:55:03.726283 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:55:03.726239    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:55:19.413952 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:55:19.413866    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:55:20.886517 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:55:20.886477    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:55:20.886947 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:55:20.886813    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 00:55:39.891224 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:55:39.891189    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:55:42.106938 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:55:42.106907    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:04.407910 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:04.407868    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:56:13.198226 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:13.198188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:15.404691 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:15.404651    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:56:22.658363 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:22.658324    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:22.759421 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:22.759385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:22.822130 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:22.822097    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:28.276263 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:28.276223    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:28.397264 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:28.397219    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:28.512303 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:28.512257    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:28.614109 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:28.614077    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:34.110524 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:34.110482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:34.210278 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:34.210237    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:56:52.308411 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:56:52.308330    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:57:01.322893 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:57:01.322852    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:57:11.805785 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:57:11.805744    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:57:19.307877 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:57:19.307838    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:57:30.507527 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:57:30.507491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:57:39.816220 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:57:39.816181    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:57:50.302380 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:57:50.302341    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:58:00.304409 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:58:00.304369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:58:21.350717 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:58:21.350624    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:58:32.004102 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:58:32.004048    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:58:52.708221 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:58:52.708186    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:59:03.910817 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:03.910777    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:59:12.148307 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:12.148238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 00:59:21.202231 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:21.202201    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:59:30.612225 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:30.612182    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:59:40.519123 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:40.519082    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:59:49.801074 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:49.800985    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 00:59:59.005159 ip-10-0-135-55 kubenswrapper[2567]: I0718 00:59:59.005120    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:00:00.141992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.141954    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"]
Jul 18 01:00:00.145461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.145440    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"
Jul 18 01:00:00.147814 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.147783    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:00:00.155743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.155722    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"
Jul 18 01:00:00.161782 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.161726    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"]
Jul 18 01:00:00.303591 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.303174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"]
Jul 18 01:00:00.309292 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:00:00.309249    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb0bb4a_d789_4fb4_babc_8f38ab93fa55.slice/crio-22c8d4a095170322e88266c6dcf7a4a6d3c2b7cceb4692419d4d1642f73af4d4 WatchSource:0}: Error finding container 22c8d4a095170322e88266c6dcf7a4a6d3c2b7cceb4692419d4d1642f73af4d4: Status 404 returned error can't find the container with id 22c8d4a095170322e88266c6dcf7a4a6d3c2b7cceb4692419d4d1642f73af4d4
Jul 18 01:00:00.311146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.311131    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:00:00.673845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:00.673809    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t" event={"ID":"ffb0bb4a-d789-4fb4-babc-8f38ab93fa55","Type":"ContainerStarted","Data":"22c8d4a095170322e88266c6dcf7a4a6d3c2b7cceb4692419d4d1642f73af4d4"}
Jul 18 01:00:04.693749 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:04.693714    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t" event={"ID":"ffb0bb4a-d789-4fb4-babc-8f38ab93fa55","Type":"ContainerStarted","Data":"0598a6eba35a97e95b7c4e8660b3576a14c2031ba782faecb9e27f3965155da2"}
Jul 18 01:00:04.707313 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:04.707124    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t" podStartSLOduration=0.683352679 podStartE2EDuration="4.707106222s" totalImagesPullingTime="4.023753543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:00:04.706808309 +0000 UTC m=+1184.234313270" watchObservedRunningTime="2026-07-18 01:00:04.707106222 +0000 UTC m=+1184.234611160"
Jul 18 01:00:05.699904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:05.699864    2567 generic.go:356] "Generic (PLEG): container finished" podID="ffb0bb4a-d789-4fb4-babc-8f38ab93fa55" containerID="0598a6eba35a97e95b7c4e8660b3576a14c2031ba782faecb9e27f3965155da2" exitCode=0
Jul 18 01:00:05.700298 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:05.699955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t" event={"ID":"ffb0bb4a-d789-4fb4-babc-8f38ab93fa55","Type":"ContainerDied","Data":"0598a6eba35a97e95b7c4e8660b3576a14c2031ba782faecb9e27f3965155da2"}
Jul 18 01:00:06.845358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:06.845332    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"
Jul 18 01:00:07.710555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:07.710518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t" event={"ID":"ffb0bb4a-d789-4fb4-babc-8f38ab93fa55","Type":"ContainerDied","Data":"22c8d4a095170322e88266c6dcf7a4a6d3c2b7cceb4692419d4d1642f73af4d4"}
Jul 18 01:00:07.710555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:07.710541    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738940-wdx9t"
Jul 18 01:00:07.710773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:07.710567    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22c8d4a095170322e88266c6dcf7a4a6d3c2b7cceb4692419d4d1642f73af4d4"
Jul 18 01:00:20.915461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:20.915431    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:00:20.916927 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:20.916900    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:00:23.871912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:23.871879    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"]
Jul 18 01:00:23.872360 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:23.872120    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c" podUID="81723458-bf74-4b7d-ab11-6466e13bcd75" containerName="manager" containerID="cri-o://1bbd2c7695a5e4e50205370c1d360a4791618a766626489ece3b480a0aa09c28" gracePeriod=10
Jul 18 01:00:26.806233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.806197    2567 generic.go:356] "Generic (PLEG): container finished" podID="81723458-bf74-4b7d-ab11-6466e13bcd75" containerID="1bbd2c7695a5e4e50205370c1d360a4791618a766626489ece3b480a0aa09c28" exitCode=0
Jul 18 01:00:26.806635 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.806295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c" event={"ID":"81723458-bf74-4b7d-ab11-6466e13bcd75","Type":"ContainerDied","Data":"1bbd2c7695a5e4e50205370c1d360a4791618a766626489ece3b480a0aa09c28"}
Jul 18 01:00:26.835926 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.835900    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 01:00:26.872204 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.872173    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81723458-bf74-4b7d-ab11-6466e13bcd75-extensions-socket-volume\") pod \"81723458-bf74-4b7d-ab11-6466e13bcd75\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") "
Jul 18 01:00:26.872404 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.872212    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghv5t\" (UniqueName: \"kubernetes.io/projected/81723458-bf74-4b7d-ab11-6466e13bcd75-kube-api-access-ghv5t\") pod \"81723458-bf74-4b7d-ab11-6466e13bcd75\" (UID: \"81723458-bf74-4b7d-ab11-6466e13bcd75\") "
Jul 18 01:00:26.872590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.872564    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81723458-bf74-4b7d-ab11-6466e13bcd75-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81723458-bf74-4b7d-ab11-6466e13bcd75" (UID: "81723458-bf74-4b7d-ab11-6466e13bcd75"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:00:26.874564 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.874537    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81723458-bf74-4b7d-ab11-6466e13bcd75-kube-api-access-ghv5t" (OuterVolumeSpecName: "kube-api-access-ghv5t") pod "81723458-bf74-4b7d-ab11-6466e13bcd75" (UID: "81723458-bf74-4b7d-ab11-6466e13bcd75"). InnerVolumeSpecName "kube-api-access-ghv5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:00:26.973974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.973926    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81723458-bf74-4b7d-ab11-6466e13bcd75-extensions-socket-volume\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:00:26.974151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:26.974073    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghv5t\" (UniqueName: \"kubernetes.io/projected/81723458-bf74-4b7d-ab11-6466e13bcd75-kube-api-access-ghv5t\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:00:27.812371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:27.812334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c" event={"ID":"81723458-bf74-4b7d-ab11-6466e13bcd75","Type":"ContainerDied","Data":"8e1fb2824b3abd75b642cfb059a354beca2f32f2ae9b09f7388e6621fcf0e0a1"}
Jul 18 01:00:27.812862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:27.812398    2567 scope.go:117] "RemoveContainer" containerID="1bbd2c7695a5e4e50205370c1d360a4791618a766626489ece3b480a0aa09c28"
Jul 18 01:00:27.812862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:27.812341    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"
Jul 18 01:00:27.829476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:27.829446    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"]
Jul 18 01:00:27.833519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:27.833485    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tx2c"]
Jul 18 01:00:28.919206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:28.919171    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81723458-bf74-4b7d-ab11-6466e13bcd75" path="/var/lib/kubelet/pods/81723458-bf74-4b7d-ab11-6466e13bcd75/volumes"
Jul 18 01:00:30.089720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:30.089689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:00:31.479873 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:00:31.479838    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:01:24.976861 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.976826    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"]
Jul 18 01:01:24.977253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.977176    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81723458-bf74-4b7d-ab11-6466e13bcd75" containerName="manager"
Jul 18 01:01:24.977253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.977188    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="81723458-bf74-4b7d-ab11-6466e13bcd75" containerName="manager"
Jul 18 01:01:24.977253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.977206    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb0bb4a-d789-4fb4-babc-8f38ab93fa55" containerName="cleanup"
Jul 18 01:01:24.977253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.977211    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb0bb4a-d789-4fb4-babc-8f38ab93fa55" containerName="cleanup"
Jul 18 01:01:24.977420 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.977286    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="81723458-bf74-4b7d-ab11-6466e13bcd75" containerName="manager"
Jul 18 01:01:24.977420 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.977298    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb0bb4a-d789-4fb4-babc-8f38ab93fa55" containerName="cleanup"
Jul 18 01:01:24.980675 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.980656    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:24.983328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.983308    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hnt59\""
Jul 18 01:01:24.991358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:24.991334    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"]
Jul 18 01:01:25.073108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.073065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aa691d8-5b45-4e33-9305-c0d67372ea18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lt4s2\" (UID: \"8aa691d8-5b45-4e33-9305-c0d67372ea18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.073299 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.073113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7tv4\" (UniqueName: \"kubernetes.io/projected/8aa691d8-5b45-4e33-9305-c0d67372ea18-kube-api-access-k7tv4\") pod \"kuadrant-operator-controller-manager-86878c769f-lt4s2\" (UID: \"8aa691d8-5b45-4e33-9305-c0d67372ea18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.174063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.174027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aa691d8-5b45-4e33-9305-c0d67372ea18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lt4s2\" (UID: \"8aa691d8-5b45-4e33-9305-c0d67372ea18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.174063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.174068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tv4\" (UniqueName: \"kubernetes.io/projected/8aa691d8-5b45-4e33-9305-c0d67372ea18-kube-api-access-k7tv4\") pod \"kuadrant-operator-controller-manager-86878c769f-lt4s2\" (UID: \"8aa691d8-5b45-4e33-9305-c0d67372ea18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.174475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.174452    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aa691d8-5b45-4e33-9305-c0d67372ea18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lt4s2\" (UID: \"8aa691d8-5b45-4e33-9305-c0d67372ea18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.183051 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.183025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7tv4\" (UniqueName: \"kubernetes.io/projected/8aa691d8-5b45-4e33-9305-c0d67372ea18-kube-api-access-k7tv4\") pod \"kuadrant-operator-controller-manager-86878c769f-lt4s2\" (UID: \"8aa691d8-5b45-4e33-9305-c0d67372ea18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.291441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.291347    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:25.439489 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:25.439465    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"]
Jul 18 01:01:25.442146 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:01:25.442119    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa691d8_5b45_4e33_9305_c0d67372ea18.slice/crio-114917c2d1988cdae9c6e87c372a7f04a04ffc9dbb7541602549632ef5887607 WatchSource:0}: Error finding container 114917c2d1988cdae9c6e87c372a7f04a04ffc9dbb7541602549632ef5887607: Status 404 returned error can't find the container with id 114917c2d1988cdae9c6e87c372a7f04a04ffc9dbb7541602549632ef5887607
Jul 18 01:01:26.096381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:26.096347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2" event={"ID":"8aa691d8-5b45-4e33-9305-c0d67372ea18","Type":"ContainerStarted","Data":"4f560fd4ee89dd975780cf07cb695a34afc7f353b9831060ddde4f2035d69810"}
Jul 18 01:01:26.096778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:26.096399    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2" event={"ID":"8aa691d8-5b45-4e33-9305-c0d67372ea18","Type":"ContainerStarted","Data":"114917c2d1988cdae9c6e87c372a7f04a04ffc9dbb7541602549632ef5887607"}
Jul 18 01:01:26.096778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:26.096441    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:26.119475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:26.119420    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2" podStartSLOduration=2.119403441 podStartE2EDuration="2.119403441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:01:26.116672779 +0000 UTC m=+1265.644177740" watchObservedRunningTime="2026-07-18 01:01:26.119403441 +0000 UTC m=+1265.646908400"
Jul 18 01:01:37.104237 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:37.104205    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt4s2"
Jul 18 01:01:46.012718 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:46.012678    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:01:48.122790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:48.122755    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:01:55.105735 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:01:55.105697    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:02:08.013394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:02:08.013354    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:02:18.906007 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:02:18.905968    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:02:30.801009 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:02:30.800970    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:02:41.107433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:02:41.107340    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:02:51.705688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:02:51.705648    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:02:55.669889 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:02:55.669835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:03:03.502815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:03:03.502774    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:03:12.909150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:03:12.909115    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:03:23.309501 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:03:23.309459    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:03:38.406978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:03:38.406939    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:03:57.902230 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:03:57.902188    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:04:04.113103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:04:04.113066    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:04:15.411537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:04:15.411443    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:04:25.005795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:04:25.005745    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:04:42.053866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:04:42.053826    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:04:42.813468 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:04:42.813435    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:04:52.109988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:04:52.109956    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:05:01.605558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:01.605519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:05:10.605634 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:10.605594    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:05:20.107662 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:20.107623    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:05:20.944358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:20.944329    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:05:20.948446 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:20.948420    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:05:29.909388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:29.909348    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:05:43.100667 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:43.100572    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:05:52.509764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:05:52.509722    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:06:02.110615 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:06:02.110576    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:06:13.606465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:06:13.606431    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:06:26.505856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:06:26.505816    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:06:39.106928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:06:39.106894    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:06:48.303248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:06:48.303209    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:07:04.807688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:04.807650    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:07:14.107128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:14.107042    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:07:16.581021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:16.580985    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:23.953839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:23.953806    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:27.510655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:27.510590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:07:32.207234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:32.207192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:36.420138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:36.420100    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:07:37.309287 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:37.309225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:40.309327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:40.309291    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:40.407983 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:40.407944    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:40.759427 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:40.759335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:46.110071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.110032    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:07:46.397076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.396979    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:46.463532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.463494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:46.567767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.567734    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"]
Jul 18 01:07:46.571726 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.571693    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.574134 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.574116    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-354f15cb-openshift-default-dockercfg-g6ml7\""
Jul 18 01:07:46.598181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.598137    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"]
Jul 18 01:07:46.603664 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.603629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:46.705558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.705464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:46.713559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713523    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-credential-socket\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.713719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713572    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-token\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.713719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713679    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-envoy\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.713856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2d9\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-kube-api-access-jz2d9\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.713856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-socket\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.713856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-certs\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.714016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713870    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/670cad49-54f9-4b53-89e4-aca56f2c4561-istiod-ca-cert\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.714016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713923    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-data\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.714016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.713951    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-podinfo\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.814982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-credential-socket\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-token\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-envoy\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2d9\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-kube-api-access-jz2d9\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815102    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-socket\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-certs\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/670cad49-54f9-4b53-89e4-aca56f2c4561-istiod-ca-cert\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-data\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815238    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-podinfo\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815511    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-credential-socket\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815708    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-data\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-certs\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.815798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.815775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-socket\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.816243 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.816219    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/670cad49-54f9-4b53-89e4-aca56f2c4561-istiod-ca-cert\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.817680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.817658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-envoy\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.817879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.817861    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-podinfo\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.825009 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.824983    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2d9\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-kube-api-access-jz2d9\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.825450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.825417    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-token\") pod \"e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") " pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:46.885005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:46.884965    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:47.034552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.034526    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"]
Jul 18 01:07:47.036700 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:07:47.036670    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod670cad49_54f9_4b53_89e4_aca56f2c4561.slice/crio-b13d1478fc77cf515dbfd6eb89ecb12f65c4a7d703e899b263f18e6f63adb028 WatchSource:0}: Error finding container b13d1478fc77cf515dbfd6eb89ecb12f65c4a7d703e899b263f18e6f63adb028: Status 404 returned error can't find the container with id b13d1478fc77cf515dbfd6eb89ecb12f65c4a7d703e899b263f18e6f63adb028
Jul 18 01:07:47.038945 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.038910    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:07:47.039028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.038984    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:07:47.039028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.039016    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:07:47.760936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.760900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:47.917495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.917457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:47.998290 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.998226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:47.998499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.998463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" event={"ID":"670cad49-54f9-4b53-89e4-aca56f2c4561","Type":"ContainerStarted","Data":"3e14f252b1c7160d87a32c140a9c70298d56f3044233357b8cf501773648622b"}
Jul 18 01:07:47.998631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:47.998532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" event={"ID":"670cad49-54f9-4b53-89e4-aca56f2c4561","Type":"ContainerStarted","Data":"b13d1478fc77cf515dbfd6eb89ecb12f65c4a7d703e899b263f18e6f63adb028"}
Jul 18 01:07:48.022483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:48.022381    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" podStartSLOduration=2.022359674 podStartE2EDuration="2.022359674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:07:48.020992069 +0000 UTC m=+1647.548497029" watchObservedRunningTime="2026-07-18 01:07:48.022359674 +0000 UTC m=+1647.549864637"
Jul 18 01:07:48.215388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:48.215347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:48.435671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:48.435633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:48.536292 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:48.536232    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:48.885433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:48.885396    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:48.890736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:48.890710    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:49.003903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:49.003878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:49.004912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:49.004895    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:07:50.349561 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:50.349519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:07:52.737169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.737121    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"]
Jul 18 01:07:52.741183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.741158    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:07:52.743531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.743508    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:07:52.743661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.743506    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:07:52.747595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.747568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"]
Jul 18 01:07:52.879992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.879941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d440f2-5160-4538-81cf-5053d2948d7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm\" (UID: \"65d440f2-5160-4538-81cf-5053d2948d7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:07:52.980691 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.980645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d440f2-5160-4538-81cf-5053d2948d7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm\" (UID: \"65d440f2-5160-4538-81cf-5053d2948d7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:07:52.981510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:52.981489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d440f2-5160-4538-81cf-5053d2948d7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm\" (UID: \"65d440f2-5160-4538-81cf-5053d2948d7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:07:53.058189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:53.057890    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:07:53.234663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:53.234636    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"]
Jul 18 01:07:53.237913 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:07:53.237870    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d440f2_5160_4538_81cf_5053d2948d7d.slice/crio-50a11481cc9b9960b8fec256188dd49c1903ad959450c9aa8a67d73841d5b9f2 WatchSource:0}: Error finding container 50a11481cc9b9960b8fec256188dd49c1903ad959450c9aa8a67d73841d5b9f2: Status 404 returned error can't find the container with id 50a11481cc9b9960b8fec256188dd49c1903ad959450c9aa8a67d73841d5b9f2
Jul 18 01:07:54.030410 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:54.030369    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerStarted","Data":"e231e26ba16791a441bcf1dcc41edf8e4c6b04f9f1d5a0881d806491a1c635dc"}
Jul 18 01:07:54.030819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:54.030432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerStarted","Data":"50a11481cc9b9960b8fec256188dd49c1903ad959450c9aa8a67d73841d5b9f2"}
Jul 18 01:07:54.048018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:54.047961    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" podStartSLOduration=2.047943382 podStartE2EDuration="2.047943382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:07:54.045907889 +0000 UTC m=+1653.573412863" watchObservedRunningTime="2026-07-18 01:07:54.047943382 +0000 UTC m=+1653.575448342"
Jul 18 01:07:55.036447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:55.036410    2567 generic.go:356] "Generic (PLEG): container finished" podID="65d440f2-5160-4538-81cf-5053d2948d7d" containerID="e231e26ba16791a441bcf1dcc41edf8e4c6b04f9f1d5a0881d806491a1c635dc" exitCode=7
Jul 18 01:07:55.036848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:55.036486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerDied","Data":"e231e26ba16791a441bcf1dcc41edf8e4c6b04f9f1d5a0881d806491a1c635dc"}
Jul 18 01:07:55.036848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:55.036732    2567 scope.go:117] "RemoveContainer" containerID="e231e26ba16791a441bcf1dcc41edf8e4c6b04f9f1d5a0881d806491a1c635dc"
Jul 18 01:07:56.043703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:56.043669    2567 generic.go:356] "Generic (PLEG): container finished" podID="65d440f2-5160-4538-81cf-5053d2948d7d" containerID="c6d37113050a3cb662ac609bceb9c8ac9db1c061807aa3d64e5c6d87a9a71f24" exitCode=7
Jul 18 01:07:56.044186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:56.043753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerDied","Data":"c6d37113050a3cb662ac609bceb9c8ac9db1c061807aa3d64e5c6d87a9a71f24"}
Jul 18 01:07:56.044186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:56.043816    2567 scope.go:117] "RemoveContainer" containerID="e231e26ba16791a441bcf1dcc41edf8e4c6b04f9f1d5a0881d806491a1c635dc"
Jul 18 01:07:56.044186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:56.043987    2567 scope.go:117] "RemoveContainer" containerID="c6d37113050a3cb662ac609bceb9c8ac9db1c061807aa3d64e5c6d87a9a71f24"
Jul 18 01:07:56.044370 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:07:56.044284    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm_odh-ai-gateway-infra(65d440f2-5160-4538-81cf-5053d2948d7d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" podUID="65d440f2-5160-4538-81cf-5053d2948d7d"
Jul 18 01:07:57.051282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:57.051222    2567 scope.go:117] "RemoveContainer" containerID="c6d37113050a3cb662ac609bceb9c8ac9db1c061807aa3d64e5c6d87a9a71f24"
Jul 18 01:07:57.051770 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:07:57.051573    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm_odh-ai-gateway-infra(65d440f2-5160-4538-81cf-5053d2948d7d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" podUID="65d440f2-5160-4538-81cf-5053d2948d7d"
Jul 18 01:07:59.070443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.070406    2567 generic.go:356] "Generic (PLEG): container finished" podID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerID="3e14f252b1c7160d87a32c140a9c70298d56f3044233357b8cf501773648622b" exitCode=0
Jul 18 01:07:59.070852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.070465    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" event={"ID":"670cad49-54f9-4b53-89e4-aca56f2c4561","Type":"ContainerDied","Data":"3e14f252b1c7160d87a32c140a9c70298d56f3044233357b8cf501773648622b"}
Jul 18 01:07:59.070893 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.070872    2567 scope.go:117] "RemoveContainer" containerID="3e14f252b1c7160d87a32c140a9c70298d56f3044233357b8cf501773648622b"
Jul 18 01:07:59.071897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.071871    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:07:59.071950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.071937    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:07:59.072036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.071969    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:07:59.208691 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:07:59.208613    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:08:00.081133 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:00.081093    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" event={"ID":"670cad49-54f9-4b53-89e4-aca56f2c4561","Type":"ContainerStarted","Data":"c4615cdf7035d940924dcddf5d11f7285e0174e857a964b252637167ca8517b5"}
Jul 18 01:08:00.885203 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:00.885162    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:08:00.886693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:00.886664    2567 patch_prober.go:28] interesting pod/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 18 01:08:00.886844 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:00.886717    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 18 01:08:01.886085 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:01.886048    2567 patch_prober.go:28] interesting pod/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 18 01:08:01.886526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:01.886101    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 18 01:08:02.885560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:02.885516    2567 patch_prober.go:28] interesting pod/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 18 01:08:02.885753 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:02.885587    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 18 01:08:03.886223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:03.886183    2567 patch_prober.go:28] interesting pod/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 18 01:08:03.886733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:03.886240    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 18 01:08:04.003968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:04.003939    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:08:04.890168 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:04.890135    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:08:04.891129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:04.891105    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:08:09.913686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:09.913653    2567 scope.go:117] "RemoveContainer" containerID="c6d37113050a3cb662ac609bceb9c8ac9db1c061807aa3d64e5c6d87a9a71f24"
Jul 18 01:08:10.133903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:10.133871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerStarted","Data":"a5023597d23dda800d366a9a26ddced69b49dce44b487f206acfcdf20a1a552e"}
Jul 18 01:08:11.140884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:11.140852    2567 generic.go:356] "Generic (PLEG): container finished" podID="65d440f2-5160-4538-81cf-5053d2948d7d" containerID="a5023597d23dda800d366a9a26ddced69b49dce44b487f206acfcdf20a1a552e" exitCode=7
Jul 18 01:08:11.141392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:11.140930    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerDied","Data":"a5023597d23dda800d366a9a26ddced69b49dce44b487f206acfcdf20a1a552e"}
Jul 18 01:08:11.141392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:11.141007    2567 scope.go:117] "RemoveContainer" containerID="c6d37113050a3cb662ac609bceb9c8ac9db1c061807aa3d64e5c6d87a9a71f24"
Jul 18 01:08:11.141392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:11.141390    2567 scope.go:117] "RemoveContainer" containerID="a5023597d23dda800d366a9a26ddced69b49dce44b487f206acfcdf20a1a552e"
Jul 18 01:08:11.141624 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:08:11.141606    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm_odh-ai-gateway-infra(65d440f2-5160-4538-81cf-5053d2948d7d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" podUID="65d440f2-5160-4538-81cf-5053d2948d7d"
Jul 18 01:08:11.174875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:11.174837    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"]
Jul 18 01:08:12.288970 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:12.288945    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:08:12.374079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:12.374044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d440f2-5160-4538-81cf-5053d2948d7d-maas-api-service-ca\") pod \"65d440f2-5160-4538-81cf-5053d2948d7d\" (UID: \"65d440f2-5160-4538-81cf-5053d2948d7d\") "
Jul 18 01:08:12.374478 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:12.374454    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65d440f2-5160-4538-81cf-5053d2948d7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65d440f2-5160-4538-81cf-5053d2948d7d" (UID: "65d440f2-5160-4538-81cf-5053d2948d7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:08:12.474790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:12.474711    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65d440f2-5160-4538-81cf-5053d2948d7d-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:08:13.154508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:13.154470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm" event={"ID":"65d440f2-5160-4538-81cf-5053d2948d7d","Type":"ContainerDied","Data":"50a11481cc9b9960b8fec256188dd49c1903ad959450c9aa8a67d73841d5b9f2"}
Jul 18 01:08:13.154508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:13.154502    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"
Jul 18 01:08:13.154762 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:13.154536    2567 scope.go:117] "RemoveContainer" containerID="a5023597d23dda800d366a9a26ddced69b49dce44b487f206acfcdf20a1a552e"
Jul 18 01:08:13.171157 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:13.171126    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"]
Jul 18 01:08:13.173339 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:13.173313    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-rw2vm"]
Jul 18 01:08:14.919766 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:14.919729    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" path="/var/lib/kubelet/pods/65d440f2-5160-4538-81cf-5053d2948d7d/volumes"
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.781665    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"]
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.782634    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.782656    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.782690    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.782698    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.782891    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:08:52.784779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.782902    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:08:52.787926 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.787328    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:08:52.790045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.790017    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:08:52.790367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.790017    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:08:52.791873 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.791846    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"]
Jul 18 01:08:52.852599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.852552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b337e67-833e-4489-89ea-0f5dc92a6096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf\" (UID: \"1b337e67-833e-4489-89ea-0f5dc92a6096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:08:52.953294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.953239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b337e67-833e-4489-89ea-0f5dc92a6096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf\" (UID: \"1b337e67-833e-4489-89ea-0f5dc92a6096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:08:52.953901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:52.953879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b337e67-833e-4489-89ea-0f5dc92a6096-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf\" (UID: \"1b337e67-833e-4489-89ea-0f5dc92a6096\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:08:53.100016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:53.099980    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:08:53.244355 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:53.244326    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"]
Jul 18 01:08:53.246860 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:08:53.246824    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b337e67_833e_4489_89ea_0f5dc92a6096.slice/crio-63d003c5726e886c509ba698655e96a1815bb6dddf562f2369774bf10f680ce4 WatchSource:0}: Error finding container 63d003c5726e886c509ba698655e96a1815bb6dddf562f2369774bf10f680ce4: Status 404 returned error can't find the container with id 63d003c5726e886c509ba698655e96a1815bb6dddf562f2369774bf10f680ce4
Jul 18 01:08:53.349901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:53.349859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" event={"ID":"1b337e67-833e-4489-89ea-0f5dc92a6096","Type":"ContainerStarted","Data":"63d003c5726e886c509ba698655e96a1815bb6dddf562f2369774bf10f680ce4"}
Jul 18 01:08:54.357303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:54.357252    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerID="c103c2271cb208dc0abbf320ada71fdc3c47909477c4d365ee62f37b7973c0ab" exitCode=7
Jul 18 01:08:54.357725 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:54.357351    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" event={"ID":"1b337e67-833e-4489-89ea-0f5dc92a6096","Type":"ContainerDied","Data":"c103c2271cb208dc0abbf320ada71fdc3c47909477c4d365ee62f37b7973c0ab"}
Jul 18 01:08:54.357725 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:54.357487    2567 scope.go:117] "RemoveContainer" containerID="c103c2271cb208dc0abbf320ada71fdc3c47909477c4d365ee62f37b7973c0ab"
Jul 18 01:08:55.363988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:55.363946    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerID="1ee5371ec8c1550fec741c6a55f9f1a6cdf6030f176d59a268c490d988413b50" exitCode=7
Jul 18 01:08:55.364513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:55.363995    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" event={"ID":"1b337e67-833e-4489-89ea-0f5dc92a6096","Type":"ContainerDied","Data":"1ee5371ec8c1550fec741c6a55f9f1a6cdf6030f176d59a268c490d988413b50"}
Jul 18 01:08:55.364513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:55.364049    2567 scope.go:117] "RemoveContainer" containerID="c103c2271cb208dc0abbf320ada71fdc3c47909477c4d365ee62f37b7973c0ab"
Jul 18 01:08:55.364513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:55.364231    2567 scope.go:117] "RemoveContainer" containerID="1ee5371ec8c1550fec741c6a55f9f1a6cdf6030f176d59a268c490d988413b50"
Jul 18 01:08:55.364513 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:08:55.364499    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf_odh-ai-gateway-infra(1b337e67-833e-4489-89ea-0f5dc92a6096)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096"
Jul 18 01:08:56.371309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:08:56.371281    2567 scope.go:117] "RemoveContainer" containerID="1ee5371ec8c1550fec741c6a55f9f1a6cdf6030f176d59a268c490d988413b50"
Jul 18 01:08:56.371683 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:08:56.371492    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf_odh-ai-gateway-infra(1b337e67-833e-4489-89ea-0f5dc92a6096)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096"
Jul 18 01:09:07.913704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:07.913667    2567 scope.go:117] "RemoveContainer" containerID="1ee5371ec8c1550fec741c6a55f9f1a6cdf6030f176d59a268c490d988413b50"
Jul 18 01:09:08.434196 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:08.434155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" event={"ID":"1b337e67-833e-4489-89ea-0f5dc92a6096","Type":"ContainerStarted","Data":"ee23b3445b96afdf3a9ccb6ec540846f29a5bdf25fa0f62f19b818a39120c936"}
Jul 18 01:09:08.448144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:08.448089    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" podStartSLOduration=16.448074135 podStartE2EDuration="16.448074135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:09:08.446798772 +0000 UTC m=+1727.974303736" watchObservedRunningTime="2026-07-18 01:09:08.448074135 +0000 UTC m=+1727.975579094"
Jul 18 01:09:09.440318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:09.440262    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerID="ee23b3445b96afdf3a9ccb6ec540846f29a5bdf25fa0f62f19b818a39120c936" exitCode=7
Jul 18 01:09:09.440770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:09.440339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" event={"ID":"1b337e67-833e-4489-89ea-0f5dc92a6096","Type":"ContainerDied","Data":"ee23b3445b96afdf3a9ccb6ec540846f29a5bdf25fa0f62f19b818a39120c936"}
Jul 18 01:09:09.440770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:09.440403    2567 scope.go:117] "RemoveContainer" containerID="1ee5371ec8c1550fec741c6a55f9f1a6cdf6030f176d59a268c490d988413b50"
Jul 18 01:09:09.440770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:09.440722    2567 scope.go:117] "RemoveContainer" containerID="ee23b3445b96afdf3a9ccb6ec540846f29a5bdf25fa0f62f19b818a39120c936"
Jul 18 01:09:09.440977 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:09:09.440953    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf_odh-ai-gateway-infra(1b337e67-833e-4489-89ea-0f5dc92a6096)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096"
Jul 18 01:09:09.465674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:09.465638    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"]
Jul 18 01:09:10.588478 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:10.588448    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:09:10.711904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:10.711809    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b337e67-833e-4489-89ea-0f5dc92a6096-maas-api-service-ca\") pod \"1b337e67-833e-4489-89ea-0f5dc92a6096\" (UID: \"1b337e67-833e-4489-89ea-0f5dc92a6096\") "
Jul 18 01:09:10.712238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:10.712205    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b337e67-833e-4489-89ea-0f5dc92a6096-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b337e67-833e-4489-89ea-0f5dc92a6096" (UID: "1b337e67-833e-4489-89ea-0f5dc92a6096"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:09:10.813316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:10.813248    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b337e67-833e-4489-89ea-0f5dc92a6096-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:09:11.453869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:11.453842    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"
Jul 18 01:09:11.454048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:11.453879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf" event={"ID":"1b337e67-833e-4489-89ea-0f5dc92a6096","Type":"ContainerDied","Data":"63d003c5726e886c509ba698655e96a1815bb6dddf562f2369774bf10f680ce4"}
Jul 18 01:09:11.454048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:11.453948    2567 scope.go:117] "RemoveContainer" containerID="ee23b3445b96afdf3a9ccb6ec540846f29a5bdf25fa0f62f19b818a39120c936"
Jul 18 01:09:11.470222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:11.470192    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"]
Jul 18 01:09:11.471835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:11.471815    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4bxrf"]
Jul 18 01:09:12.919529 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:12.919496    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" path="/var/lib/kubelet/pods/1b337e67-833e-4489-89ea-0f5dc92a6096/volumes"
Jul 18 01:09:42.825208 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825173    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"]
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825547    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825558    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825576    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825581    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825592    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825597    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825606    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825611    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825666    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d440f2-5160-4538-81cf-5053d2948d7d" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825675    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.825716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.825682    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:09:42.829982 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.829962    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:42.832289 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.832180    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:09:42.832505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.832452    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:09:42.835949 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.835923    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"]
Jul 18 01:09:42.911720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:42.911675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc\" (UID: \"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:43.012692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.012650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc\" (UID: \"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:43.013376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.013352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc\" (UID: \"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:43.143105 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.143012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:43.301654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.301627    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"]
Jul 18 01:09:43.303389 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:09:43.303358    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ee68e9_7a5e_4f2b_bb11_7b233d1d3e22.slice/crio-2c94b1423e0e6c7e97ce562f48a375411d235884280afd963583d8436b0a680c WatchSource:0}: Error finding container 2c94b1423e0e6c7e97ce562f48a375411d235884280afd963583d8436b0a680c: Status 404 returned error can't find the container with id 2c94b1423e0e6c7e97ce562f48a375411d235884280afd963583d8436b0a680c
Jul 18 01:09:43.622150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.622103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" event={"ID":"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22","Type":"ContainerStarted","Data":"865b9bf383f91d1059a326a3160bebb42d00c72a3c2ec2d33deec97dad6e921e"}
Jul 18 01:09:43.622428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.622178    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" event={"ID":"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22","Type":"ContainerStarted","Data":"2c94b1423e0e6c7e97ce562f48a375411d235884280afd963583d8436b0a680c"}
Jul 18 01:09:43.639861 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:43.639804    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" podStartSLOduration=1.639788223 podStartE2EDuration="1.639788223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:09:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:09:43.637019697 +0000 UTC m=+1763.164524659" watchObservedRunningTime="2026-07-18 01:09:43.639788223 +0000 UTC m=+1763.167293211"
Jul 18 01:09:44.628617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:44.628583    2567 generic.go:356] "Generic (PLEG): container finished" podID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerID="865b9bf383f91d1059a326a3160bebb42d00c72a3c2ec2d33deec97dad6e921e" exitCode=7
Jul 18 01:09:44.629030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:44.628665    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" event={"ID":"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22","Type":"ContainerDied","Data":"865b9bf383f91d1059a326a3160bebb42d00c72a3c2ec2d33deec97dad6e921e"}
Jul 18 01:09:44.629030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:44.628884    2567 scope.go:117] "RemoveContainer" containerID="865b9bf383f91d1059a326a3160bebb42d00c72a3c2ec2d33deec97dad6e921e"
Jul 18 01:09:45.635459 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:45.635367    2567 generic.go:356] "Generic (PLEG): container finished" podID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerID="a7514953df1e23a746b932ec68deefbfb7ea2bd650bc2ae44ea117498a8cfc01" exitCode=7
Jul 18 01:09:45.635891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:45.635461    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" event={"ID":"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22","Type":"ContainerDied","Data":"a7514953df1e23a746b932ec68deefbfb7ea2bd650bc2ae44ea117498a8cfc01"}
Jul 18 01:09:45.635891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:45.635530    2567 scope.go:117] "RemoveContainer" containerID="865b9bf383f91d1059a326a3160bebb42d00c72a3c2ec2d33deec97dad6e921e"
Jul 18 01:09:45.635891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:45.635651    2567 scope.go:117] "RemoveContainer" containerID="a7514953df1e23a746b932ec68deefbfb7ea2bd650bc2ae44ea117498a8cfc01"
Jul 18 01:09:45.635891 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:09:45.635818    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc_odh-ai-gateway-infra(01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22"
Jul 18 01:09:46.641897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:46.641871    2567 scope.go:117] "RemoveContainer" containerID="a7514953df1e23a746b932ec68deefbfb7ea2bd650bc2ae44ea117498a8cfc01"
Jul 18 01:09:46.642297 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:09:46.642099    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc_odh-ai-gateway-infra(01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22"
Jul 18 01:09:56.913094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:56.913065    2567 scope.go:117] "RemoveContainer" containerID="a7514953df1e23a746b932ec68deefbfb7ea2bd650bc2ae44ea117498a8cfc01"
Jul 18 01:09:57.700229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:57.700144    2567 generic.go:356] "Generic (PLEG): container finished" podID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerID="7d39648ab333b0b5dbbc29ad013f7b0d490153dd2591412308d74b8d98b945b4" exitCode=7
Jul 18 01:09:57.700229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:57.700215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" event={"ID":"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22","Type":"ContainerDied","Data":"7d39648ab333b0b5dbbc29ad013f7b0d490153dd2591412308d74b8d98b945b4"}
Jul 18 01:09:57.700434 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:57.700306    2567 scope.go:117] "RemoveContainer" containerID="a7514953df1e23a746b932ec68deefbfb7ea2bd650bc2ae44ea117498a8cfc01"
Jul 18 01:09:57.700632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:57.700611    2567 scope.go:117] "RemoveContainer" containerID="7d39648ab333b0b5dbbc29ad013f7b0d490153dd2591412308d74b8d98b945b4"
Jul 18 01:09:57.700857 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:09:57.700835    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc_odh-ai-gateway-infra(01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22"
Jul 18 01:09:58.724731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:58.724698    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"]
Jul 18 01:09:58.867333 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:58.867301    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:58.949319 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:58.949256    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22-maas-api-service-ca\") pod \"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22\" (UID: \"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22\") "
Jul 18 01:09:58.949625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:58.949601    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" (UID: "01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:09:59.050321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:59.050204    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:09:59.713538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:59.713503    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc" event={"ID":"01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22","Type":"ContainerDied","Data":"2c94b1423e0e6c7e97ce562f48a375411d235884280afd963583d8436b0a680c"}
Jul 18 01:09:59.713734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:59.713568    2567 scope.go:117] "RemoveContainer" containerID="7d39648ab333b0b5dbbc29ad013f7b0d490153dd2591412308d74b8d98b945b4"
Jul 18 01:09:59.713734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:59.713514    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"
Jul 18 01:09:59.736860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:59.736833    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"]
Jul 18 01:09:59.741405 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:09:59.741380    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j4qlc"]
Jul 18 01:10:00.919171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:00.919134    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" path="/var/lib/kubelet/pods/01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22/volumes"
Jul 18 01:10:20.975122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:20.975085    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:10:20.981533 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:20.981502    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:10:32.876492 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876461    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"]
Jul 18 01:10:32.876877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876825    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:10:32.876877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876836    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:10:32.876877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876849    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:10:32.876877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876855    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:10:32.877001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876928    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b337e67-833e-4489-89ea-0f5dc92a6096" containerName="revoke-keys"
Jul 18 01:10:32.877001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876938    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:10:32.877001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.876944    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:10:32.880363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.880346    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:32.882785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.882760    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:10:32.882785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.882760    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:10:32.887537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.887509    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"]
Jul 18 01:10:32.967535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:32.967495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68c50b9-e7c7-462d-bf6b-292d7b501e0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr\" (UID: \"b68c50b9-e7c7-462d-bf6b-292d7b501e0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:33.068692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.068642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68c50b9-e7c7-462d-bf6b-292d7b501e0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr\" (UID: \"b68c50b9-e7c7-462d-bf6b-292d7b501e0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:33.069481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.069458    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68c50b9-e7c7-462d-bf6b-292d7b501e0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr\" (UID: \"b68c50b9-e7c7-462d-bf6b-292d7b501e0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:33.192718 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.192627    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:33.332765 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.332736    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"]
Jul 18 01:10:33.334486 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:10:33.334447    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68c50b9_e7c7_462d_bf6b_292d7b501e0d.slice/crio-24e8c72c96ae3a620b9ce73b56267138c90629ff1675b571b4045f0c90700c6d WatchSource:0}: Error finding container 24e8c72c96ae3a620b9ce73b56267138c90629ff1675b571b4045f0c90700c6d: Status 404 returned error can't find the container with id 24e8c72c96ae3a620b9ce73b56267138c90629ff1675b571b4045f0c90700c6d
Jul 18 01:10:33.883483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.883396    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" event={"ID":"b68c50b9-e7c7-462d-bf6b-292d7b501e0d","Type":"ContainerStarted","Data":"c64b335e83439fc0a5e1f8f2705c30bd5e384b096e96cd84b90a2a0eae2decb0"}
Jul 18 01:10:33.883483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.883445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" event={"ID":"b68c50b9-e7c7-462d-bf6b-292d7b501e0d","Type":"ContainerStarted","Data":"24e8c72c96ae3a620b9ce73b56267138c90629ff1675b571b4045f0c90700c6d"}
Jul 18 01:10:33.901004 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:33.900945    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" podStartSLOduration=1.900927729 podStartE2EDuration="1.900927729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:10:33.899685211 +0000 UTC m=+1813.427190171" watchObservedRunningTime="2026-07-18 01:10:33.900927729 +0000 UTC m=+1813.428432688"
Jul 18 01:10:34.889514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:34.889481    2567 generic.go:356] "Generic (PLEG): container finished" podID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerID="c64b335e83439fc0a5e1f8f2705c30bd5e384b096e96cd84b90a2a0eae2decb0" exitCode=7
Jul 18 01:10:34.889962 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:34.889562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" event={"ID":"b68c50b9-e7c7-462d-bf6b-292d7b501e0d","Type":"ContainerDied","Data":"c64b335e83439fc0a5e1f8f2705c30bd5e384b096e96cd84b90a2a0eae2decb0"}
Jul 18 01:10:34.889962 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:34.889704    2567 scope.go:117] "RemoveContainer" containerID="c64b335e83439fc0a5e1f8f2705c30bd5e384b096e96cd84b90a2a0eae2decb0"
Jul 18 01:10:35.896374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:35.896338    2567 generic.go:356] "Generic (PLEG): container finished" podID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerID="71d53452f926e15b034f3a12869d1b2ed9ff804ceec44d3ef37073b10d2e5c07" exitCode=7
Jul 18 01:10:35.896779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:35.896425    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" event={"ID":"b68c50b9-e7c7-462d-bf6b-292d7b501e0d","Type":"ContainerDied","Data":"71d53452f926e15b034f3a12869d1b2ed9ff804ceec44d3ef37073b10d2e5c07"}
Jul 18 01:10:35.896779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:35.896488    2567 scope.go:117] "RemoveContainer" containerID="c64b335e83439fc0a5e1f8f2705c30bd5e384b096e96cd84b90a2a0eae2decb0"
Jul 18 01:10:35.896779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:35.896726    2567 scope.go:117] "RemoveContainer" containerID="71d53452f926e15b034f3a12869d1b2ed9ff804ceec44d3ef37073b10d2e5c07"
Jul 18 01:10:35.896965 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:10:35.896942    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr_odh-ai-gateway-infra(b68c50b9-e7c7-462d-bf6b-292d7b501e0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d"
Jul 18 01:10:36.902852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:36.902826    2567 scope.go:117] "RemoveContainer" containerID="71d53452f926e15b034f3a12869d1b2ed9ff804ceec44d3ef37073b10d2e5c07"
Jul 18 01:10:36.903263 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:10:36.902998    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr_odh-ai-gateway-infra(b68c50b9-e7c7-462d-bf6b-292d7b501e0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d"
Jul 18 01:10:49.913771 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:49.913735    2567 scope.go:117] "RemoveContainer" containerID="71d53452f926e15b034f3a12869d1b2ed9ff804ceec44d3ef37073b10d2e5c07"
Jul 18 01:10:50.972589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:50.972553    2567 generic.go:356] "Generic (PLEG): container finished" podID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerID="f405bd1448238b538afa27f6e57ae160d4e6d17cfe776f89d3b761a133fc4e28" exitCode=7
Jul 18 01:10:50.973079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:50.972637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" event={"ID":"b68c50b9-e7c7-462d-bf6b-292d7b501e0d","Type":"ContainerDied","Data":"f405bd1448238b538afa27f6e57ae160d4e6d17cfe776f89d3b761a133fc4e28"}
Jul 18 01:10:50.973079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:50.972714    2567 scope.go:117] "RemoveContainer" containerID="71d53452f926e15b034f3a12869d1b2ed9ff804ceec44d3ef37073b10d2e5c07"
Jul 18 01:10:50.973079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:50.973048    2567 scope.go:117] "RemoveContainer" containerID="f405bd1448238b538afa27f6e57ae160d4e6d17cfe776f89d3b761a133fc4e28"
Jul 18 01:10:50.973347 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:10:50.973326    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr_odh-ai-gateway-infra(b68c50b9-e7c7-462d-bf6b-292d7b501e0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d"
Jul 18 01:10:51.997146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:51.997109    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"]
Jul 18 01:10:52.140554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.140530    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:52.245155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.245118    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68c50b9-e7c7-462d-bf6b-292d7b501e0d-maas-api-service-ca\") pod \"b68c50b9-e7c7-462d-bf6b-292d7b501e0d\" (UID: \"b68c50b9-e7c7-462d-bf6b-292d7b501e0d\") "
Jul 18 01:10:52.245509 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.245486    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b68c50b9-e7c7-462d-bf6b-292d7b501e0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b68c50b9-e7c7-462d-bf6b-292d7b501e0d" (UID: "b68c50b9-e7c7-462d-bf6b-292d7b501e0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:10:52.346090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.346048    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b68c50b9-e7c7-462d-bf6b-292d7b501e0d-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:52.859361 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.859316    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"]
Jul 18 01:10:52.859612 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.859568    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy" containerID="cri-o://c4615cdf7035d940924dcddf5d11f7285e0174e857a964b252637167ca8517b5" gracePeriod=30
Jul 18 01:10:52.915586 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.915553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:10:52.987183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.987148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:10:52.987933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.987900    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr" event={"ID":"b68c50b9-e7c7-462d-bf6b-292d7b501e0d","Type":"ContainerDied","Data":"24e8c72c96ae3a620b9ce73b56267138c90629ff1675b571b4045f0c90700c6d"}
Jul 18 01:10:52.988081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.987967    2567 scope.go:117] "RemoveContainer" containerID="f405bd1448238b538afa27f6e57ae160d4e6d17cfe776f89d3b761a133fc4e28"
Jul 18 01:10:52.988081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:52.987994    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"
Jul 18 01:10:53.005401 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:53.005262    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"]
Jul 18 01:10:53.007717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:53.007689    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-kbfjr"]
Jul 18 01:10:54.923290 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:54.920973    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" path="/var/lib/kubelet/pods/b68c50b9-e7c7-462d-bf6b-292d7b501e0d/volumes"
Jul 18 01:10:55.712499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:55.712458    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:10:58.016423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.016388    2567 generic.go:356] "Generic (PLEG): container finished" podID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerID="c4615cdf7035d940924dcddf5d11f7285e0174e857a964b252637167ca8517b5" exitCode=0
Jul 18 01:10:58.016867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.016451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" event={"ID":"670cad49-54f9-4b53-89e4-aca56f2c4561","Type":"ContainerDied","Data":"c4615cdf7035d940924dcddf5d11f7285e0174e857a964b252637167ca8517b5"}
Jul 18 01:10:58.016867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.016525    2567 scope.go:117] "RemoveContainer" containerID="3e14f252b1c7160d87a32c140a9c70298d56f3044233357b8cf501773648622b"
Jul 18 01:10:58.124331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.124248    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:10:58.201390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201361    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-data\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-envoy\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201425    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-certs\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201448    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-podinfo\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201506    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-token\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201535    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz2d9\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-kube-api-access-jz2d9\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201569    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-credential-socket\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201613    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-socket\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.201743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201636    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/670cad49-54f9-4b53-89e4-aca56f2c4561-istiod-ca-cert\") pod \"670cad49-54f9-4b53-89e4-aca56f2c4561\" (UID: \"670cad49-54f9-4b53-89e4-aca56f2c4561\") "
Jul 18 01:10:58.202220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.201936    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:10:58.202220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.202050    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:10:58.202220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.202185    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:10:58.202220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.202217    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-data" (OuterVolumeSpecName: "istio-data") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:10:58.202535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.202514    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/670cad49-54f9-4b53-89e4-aca56f2c4561-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:10:58.203885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.203857    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:10:58.204019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.203998    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:10:58.204419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.204401    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-token" (OuterVolumeSpecName: "istio-token") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:10:58.204524 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.204500    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-kube-api-access-jz2d9" (OuterVolumeSpecName: "kube-api-access-jz2d9") pod "670cad49-54f9-4b53-89e4-aca56f2c4561" (UID: "670cad49-54f9-4b53-89e4-aca56f2c4561"). InnerVolumeSpecName "kube-api-access-jz2d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:10:58.302879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302844    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.302879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302878    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz2d9\" (UniqueName: \"kubernetes.io/projected/670cad49-54f9-4b53-89e4-aca56f2c4561-kube-api-access-jz2d9\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302894    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302906    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302918    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/670cad49-54f9-4b53-89e4-aca56f2c4561-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302930    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302939    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302950    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/670cad49-54f9-4b53-89e4-aca56f2c4561-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.303112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.302961    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/670cad49-54f9-4b53-89e4-aca56f2c4561-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:10:58.312616 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.312584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:10:58.722307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:58.722212    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:10:59.022752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:59.022672    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"
Jul 18 01:10:59.022752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:59.022689    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx" event={"ID":"670cad49-54f9-4b53-89e4-aca56f2c4561","Type":"ContainerDied","Data":"b13d1478fc77cf515dbfd6eb89ecb12f65c4a7d703e899b263f18e6f63adb028"}
Jul 18 01:10:59.022752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:59.022749    2567 scope.go:117] "RemoveContainer" containerID="c4615cdf7035d940924dcddf5d11f7285e0174e857a964b252637167ca8517b5"
Jul 18 01:10:59.040020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:59.039994    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"]
Jul 18 01:10:59.043856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:10:59.043831    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-354f15cb-openshift-default-64bc8f8d45-bnwkx"]
Jul 18 01:11:00.925888 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:00.925857    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" path="/var/lib/kubelet/pods/670cad49-54f9-4b53-89e4-aca56f2c4561/volumes"
Jul 18 01:11:04.160105 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.160073    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:04.261000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.260968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:04.419733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.419652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:04.519477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.519441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:04.585795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.585747    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"]
Jul 18 01:11:04.586400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586378    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586404    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586420    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586429    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586440    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:11:04.586475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586449    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:11:04.586475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586471    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586480    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586494    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586502    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586594    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586607    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586616    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ee68e9-7a5e-4f2b-bb11-7b233d1d3e22" containerName="revoke-keys"
Jul 18 01:11:04.586666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.586631    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:04.591596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.591572    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.594073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.594045    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1b6716ba-gw-openshift-default-dockercfg-6jmhm\""
Jul 18 01:11:04.600953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.600926    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"]
Jul 18 01:11:04.618787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.618746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:04.659692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf99p\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-kube-api-access-hf99p\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.659899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-socket\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.659899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-envoy\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.659899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-certs\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.659899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-data\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.660124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-podinfo\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.660124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.659940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istiod-ca-cert\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.660124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.660024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-credential-socket\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.660124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.660082    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-token\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.760834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf99p\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-kube-api-access-hf99p\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.760834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-socket\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.760834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760794    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-envoy\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.760834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-certs\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.760834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-data\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-podinfo\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istiod-ca-cert\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-credential-socket\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.760999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-token\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.761407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-socket\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.761441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-certs\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.761500    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-data\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.761709    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-credential-socket\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.761945 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.761927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istiod-ca-cert\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.763342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.763315    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-envoy\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.763973 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.763950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-podinfo\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.774483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.774455    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-token\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.774695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.774668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf99p\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-kube-api-access-hf99p\") pod \"e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") " pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:04.904649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:04.904602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:05.052361 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.052326    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"]
Jul 18 01:11:05.055192 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:11:05.055164    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d248a30_f020_4dfd_9ae4_d15eaa834ea7.slice/crio-b86ffb1a5a1e33dcd4c4ce33722dfc97f60b5d8c9b718c396fa32200c8abc4ea WatchSource:0}: Error finding container b86ffb1a5a1e33dcd4c4ce33722dfc97f60b5d8c9b718c396fa32200c8abc4ea: Status 404 returned error can't find the container with id b86ffb1a5a1e33dcd4c4ce33722dfc97f60b5d8c9b718c396fa32200c8abc4ea
Jul 18 01:11:05.057308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.057240    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:11:05.057409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.057355    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:11:05.057409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.057396    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:11:05.464929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.464894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:05.565469 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.565437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:05.944188 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:05.944157    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:06.055845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.055786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:06.062330 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.062297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" event={"ID":"6d248a30-f020-4dfd-9ae4-d15eaa834ea7","Type":"ContainerStarted","Data":"ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"}
Jul 18 01:11:06.062519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.062349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" event={"ID":"6d248a30-f020-4dfd-9ae4-d15eaa834ea7","Type":"ContainerStarted","Data":"b86ffb1a5a1e33dcd4c4ce33722dfc97f60b5d8c9b718c396fa32200c8abc4ea"}
Jul 18 01:11:06.085770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.085699    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" podStartSLOduration=2.08567877 podStartE2EDuration="2.08567877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:11:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:11:06.081499437 +0000 UTC m=+1845.609004398" watchObservedRunningTime="2026-07-18 01:11:06.08567877 +0000 UTC m=+1845.613183731"
Jul 18 01:11:06.453569 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.453533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:06.554440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.554397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:06.655479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.655448    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:06.905402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.905356    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:06.910776 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:06.910747    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:07.067867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:07.067837    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:07.068850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:07.068833    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:08.505997 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:08.505961    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:11:10.420457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.420422    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"]
Jul 18 01:11:10.420894 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.420801    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy"
Jul 18 01:11:10.420894 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.420813    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy"
Jul 18 01:11:10.420894 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.420879    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="670cad49-54f9-4b53-89e4-aca56f2c4561" containerName="istio-proxy"
Jul 18 01:11:10.420894 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.420886    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b68c50b9-e7c7-462d-bf6b-292d7b501e0d" containerName="revoke-keys"
Jul 18 01:11:10.424241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.424223    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:10.426442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.426418    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:11:10.426564 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.426421    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:11:10.430612 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.430588    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"]
Jul 18 01:11:10.513917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.513874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8d1595-159a-4dc4-8b4f-359e24a4d2ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9\" (UID: \"df8d1595-159a-4dc4-8b4f-359e24a4d2ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:10.614590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.614548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8d1595-159a-4dc4-8b4f-359e24a4d2ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9\" (UID: \"df8d1595-159a-4dc4-8b4f-359e24a4d2ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:10.615213 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.615193    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8d1595-159a-4dc4-8b4f-359e24a4d2ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9\" (UID: \"df8d1595-159a-4dc4-8b4f-359e24a4d2ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:10.736313 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:10.736186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:11.130038 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:11.130013    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"]
Jul 18 01:11:11.132485 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:11:11.132457    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8d1595_159a_4dc4_8b4f_359e24a4d2ca.slice/crio-398cc471b4f761b3141c14e279c868ee825439108325755dbb9ab2310617c95e WatchSource:0}: Error finding container 398cc471b4f761b3141c14e279c868ee825439108325755dbb9ab2310617c95e: Status 404 returned error can't find the container with id 398cc471b4f761b3141c14e279c868ee825439108325755dbb9ab2310617c95e
Jul 18 01:11:12.094999 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:12.094946    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerStarted","Data":"fbbc8f2640caac93061c6609cab197870e24ce509cd71d037667d3a798015b14"}
Jul 18 01:11:12.095417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:12.095061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerStarted","Data":"398cc471b4f761b3141c14e279c868ee825439108325755dbb9ab2310617c95e"}
Jul 18 01:11:12.110432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:12.110375    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" podStartSLOduration=2.110358349 podStartE2EDuration="2.110358349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:11:12.108953159 +0000 UTC m=+1851.636458122" watchObservedRunningTime="2026-07-18 01:11:12.110358349 +0000 UTC m=+1851.637863308"
Jul 18 01:11:13.101742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:13.101699    2567 generic.go:356] "Generic (PLEG): container finished" podID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerID="fbbc8f2640caac93061c6609cab197870e24ce509cd71d037667d3a798015b14" exitCode=7
Jul 18 01:11:13.102121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:13.101771    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerDied","Data":"fbbc8f2640caac93061c6609cab197870e24ce509cd71d037667d3a798015b14"}
Jul 18 01:11:13.102121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:13.101922    2567 scope.go:117] "RemoveContainer" containerID="fbbc8f2640caac93061c6609cab197870e24ce509cd71d037667d3a798015b14"
Jul 18 01:11:14.109000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:14.108966    2567 generic.go:356] "Generic (PLEG): container finished" podID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerID="1244e6181bbf587b680eb2f9f59866a61c160825542264a100c80ecc9b6ab567" exitCode=7
Jul 18 01:11:14.109430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:14.109023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerDied","Data":"1244e6181bbf587b680eb2f9f59866a61c160825542264a100c80ecc9b6ab567"}
Jul 18 01:11:14.109430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:14.109075    2567 scope.go:117] "RemoveContainer" containerID="fbbc8f2640caac93061c6609cab197870e24ce509cd71d037667d3a798015b14"
Jul 18 01:11:14.109430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:14.109306    2567 scope.go:117] "RemoveContainer" containerID="1244e6181bbf587b680eb2f9f59866a61c160825542264a100c80ecc9b6ab567"
Jul 18 01:11:14.109572 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:11:14.109536    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9_odh-ai-gateway-infra(df8d1595-159a-4dc4-8b4f-359e24a4d2ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca"
Jul 18 01:11:15.116827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:15.116801    2567 scope.go:117] "RemoveContainer" containerID="1244e6181bbf587b680eb2f9f59866a61c160825542264a100c80ecc9b6ab567"
Jul 18 01:11:15.117332 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:11:15.117097    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9_odh-ai-gateway-infra(df8d1595-159a-4dc4-8b4f-359e24a4d2ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca"
Jul 18 01:11:15.579345 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:15.579305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:17.136247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:17.136211    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerID="ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4" exitCode=0
Jul 18 01:11:17.136705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:17.136300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" event={"ID":"6d248a30-f020-4dfd-9ae4-d15eaa834ea7","Type":"ContainerDied","Data":"ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"}
Jul 18 01:11:17.136705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:17.136698    2567 scope.go:117] "RemoveContainer" containerID="ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"
Jul 18 01:11:17.137716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:17.137686    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:11:17.137831 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:17.137764    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:11:17.137831 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:17.137798    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:11:18.145485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:18.145445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" event={"ID":"6d248a30-f020-4dfd-9ae4-d15eaa834ea7","Type":"ContainerStarted","Data":"c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18"}
Jul 18 01:11:18.905815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:18.905772    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:18.907059 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:18.907029    2567 patch_prober.go:28] interesting pod/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 18 01:11:18.907199 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:18.907082    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 18 01:11:19.905810 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:19.905771    2567 patch_prober.go:28] interesting pod/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 18 01:11:19.906324 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:19.905839    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 18 01:11:20.905631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:20.905599    2567 patch_prober.go:28] interesting pod/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 18 01:11:20.905891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:20.905655    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 18 01:11:21.905194 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:21.905149    2567 patch_prober.go:28] interesting pod/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 18 01:11:21.905438 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:21.905216    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 18 01:11:22.068492 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:22.068448    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:22.909587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:22.909559    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:22.910631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:22.910609    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:11:27.913410 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:27.913376    2567 scope.go:117] "RemoveContainer" containerID="1244e6181bbf587b680eb2f9f59866a61c160825542264a100c80ecc9b6ab567"
Jul 18 01:11:28.204335 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:28.204221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerStarted","Data":"759c642f80339bbf5988c6291ef43da867637e17c0e976d8dc5390d2f97e9e4b"}
Jul 18 01:11:29.211957 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:29.211926    2567 generic.go:356] "Generic (PLEG): container finished" podID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerID="759c642f80339bbf5988c6291ef43da867637e17c0e976d8dc5390d2f97e9e4b" exitCode=7
Jul 18 01:11:29.212398 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:29.212018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerDied","Data":"759c642f80339bbf5988c6291ef43da867637e17c0e976d8dc5390d2f97e9e4b"}
Jul 18 01:11:29.212398 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:29.212091    2567 scope.go:117] "RemoveContainer" containerID="1244e6181bbf587b680eb2f9f59866a61c160825542264a100c80ecc9b6ab567"
Jul 18 01:11:29.212506 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:29.212430    2567 scope.go:117] "RemoveContainer" containerID="759c642f80339bbf5988c6291ef43da867637e17c0e976d8dc5390d2f97e9e4b"
Jul 18 01:11:29.212658 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:11:29.212639    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9_odh-ai-gateway-infra(df8d1595-159a-4dc4-8b4f-359e24a4d2ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca"
Jul 18 01:11:29.226974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:29.226947    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"]
Jul 18 01:11:30.363352 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:30.363325    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:30.508347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:30.508207    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8d1595-159a-4dc4-8b4f-359e24a4d2ca-maas-api-service-ca\") pod \"df8d1595-159a-4dc4-8b4f-359e24a4d2ca\" (UID: \"df8d1595-159a-4dc4-8b4f-359e24a4d2ca\") "
Jul 18 01:11:30.508654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:30.508632    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8d1595-159a-4dc4-8b4f-359e24a4d2ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df8d1595-159a-4dc4-8b4f-359e24a4d2ca" (UID: "df8d1595-159a-4dc4-8b4f-359e24a4d2ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:11:30.609263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:30.609226    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8d1595-159a-4dc4-8b4f-359e24a4d2ca-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:11:31.224581 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:31.224481    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"
Jul 18 01:11:31.224581 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:31.224510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9" event={"ID":"df8d1595-159a-4dc4-8b4f-359e24a4d2ca","Type":"ContainerDied","Data":"398cc471b4f761b3141c14e279c868ee825439108325755dbb9ab2310617c95e"}
Jul 18 01:11:31.224819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:31.224587    2567 scope.go:117] "RemoveContainer" containerID="759c642f80339bbf5988c6291ef43da867637e17c0e976d8dc5390d2f97e9e4b"
Jul 18 01:11:31.240630 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:31.240602    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"]
Jul 18 01:11:31.248643 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:31.248608    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-428n9"]
Jul 18 01:11:32.918922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.918885    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" path="/var/lib/kubelet/pods/df8d1595-159a-4dc4-8b4f-359e24a4d2ca/volumes"
Jul 18 01:11:32.921564 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.921537    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"]
Jul 18 01:11:32.921946 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.921932    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.921991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.921948    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.921991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.921966    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.921991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.921972    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.922089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.922036    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.922089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.922043    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.922089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.922049    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:11:32.926416 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.926395    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:32.928715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.928696    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:11:32.928791 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.928706    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:11:32.934005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:32.933952    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"]
Jul 18 01:11:33.033243 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:33.033195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839f2c34-4010-4831-96d4-f6b04f09e5ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6\" (UID: \"839f2c34-4010-4831-96d4-f6b04f09e5ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:33.134460 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:33.134424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839f2c34-4010-4831-96d4-f6b04f09e5ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6\" (UID: \"839f2c34-4010-4831-96d4-f6b04f09e5ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:33.135104 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:33.135077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839f2c34-4010-4831-96d4-f6b04f09e5ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6\" (UID: \"839f2c34-4010-4831-96d4-f6b04f09e5ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:33.238555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:33.238456    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:33.377522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:33.377488    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"]
Jul 18 01:11:33.380209 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:11:33.380180    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839f2c34_4010_4831_96d4_f6b04f09e5ed.slice/crio-7c9cf6cdb513cc49851576d1c187e6d1406db2c12cb07195c86e2195703e98bb WatchSource:0}: Error finding container 7c9cf6cdb513cc49851576d1c187e6d1406db2c12cb07195c86e2195703e98bb: Status 404 returned error can't find the container with id 7c9cf6cdb513cc49851576d1c187e6d1406db2c12cb07195c86e2195703e98bb
Jul 18 01:11:34.241721 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:34.241685    2567 generic.go:356] "Generic (PLEG): container finished" podID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerID="df1227b1df497a74a94d66ca0123fde2c21af895ba20a0006e302c2570a8729c" exitCode=6
Jul 18 01:11:34.242138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:34.241737    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" event={"ID":"839f2c34-4010-4831-96d4-f6b04f09e5ed","Type":"ContainerDied","Data":"df1227b1df497a74a94d66ca0123fde2c21af895ba20a0006e302c2570a8729c"}
Jul 18 01:11:34.242138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:34.241796    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" event={"ID":"839f2c34-4010-4831-96d4-f6b04f09e5ed","Type":"ContainerStarted","Data":"7c9cf6cdb513cc49851576d1c187e6d1406db2c12cb07195c86e2195703e98bb"}
Jul 18 01:11:34.242138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:34.242036    2567 scope.go:117] "RemoveContainer" containerID="df1227b1df497a74a94d66ca0123fde2c21af895ba20a0006e302c2570a8729c"
Jul 18 01:11:34.787741 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:34.787703    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:34.839400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:34.839353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:35.247931 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:35.247898    2567 generic.go:356] "Generic (PLEG): container finished" podID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerID="ca56ffc4b8ad48b3a46cafbfc3dbe1d0bfe2a2c47656e9b179b11dc6c33d9d9e" exitCode=6
Jul 18 01:11:35.248422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:35.247991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" event={"ID":"839f2c34-4010-4831-96d4-f6b04f09e5ed","Type":"ContainerDied","Data":"ca56ffc4b8ad48b3a46cafbfc3dbe1d0bfe2a2c47656e9b179b11dc6c33d9d9e"}
Jul 18 01:11:35.248422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:35.248060    2567 scope.go:117] "RemoveContainer" containerID="df1227b1df497a74a94d66ca0123fde2c21af895ba20a0006e302c2570a8729c"
Jul 18 01:11:35.248422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:35.248196    2567 scope.go:117] "RemoveContainer" containerID="ca56ffc4b8ad48b3a46cafbfc3dbe1d0bfe2a2c47656e9b179b11dc6c33d9d9e"
Jul 18 01:11:35.248564 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:11:35.248436    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6_odh-ai-gateway-infra(839f2c34-4010-4831-96d4-f6b04f09e5ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed"
Jul 18 01:11:36.255507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:36.255485    2567 scope.go:117] "RemoveContainer" containerID="ca56ffc4b8ad48b3a46cafbfc3dbe1d0bfe2a2c47656e9b179b11dc6c33d9d9e"
Jul 18 01:11:36.255911 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:11:36.255698    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6_odh-ai-gateway-infra(839f2c34-4010-4831-96d4-f6b04f09e5ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed"
Jul 18 01:11:38.176510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:38.176471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:48.913154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:48.913053    2567 scope.go:117] "RemoveContainer" containerID="ca56ffc4b8ad48b3a46cafbfc3dbe1d0bfe2a2c47656e9b179b11dc6c33d9d9e"
Jul 18 01:11:49.320747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:49.320713    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" event={"ID":"839f2c34-4010-4831-96d4-f6b04f09e5ed","Type":"ContainerStarted","Data":"2ca5bde32be8d76191f4c6ece5b2299799cae1247d7c6565a0e3d9eb9e9fa7d8"}
Jul 18 01:11:49.337993 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:49.337934    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" podStartSLOduration=17.337918758 podStartE2EDuration="17.337918758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:11:49.33694951 +0000 UTC m=+1888.864454470" watchObservedRunningTime="2026-07-18 01:11:49.337918758 +0000 UTC m=+1888.865423718"
Jul 18 01:11:50.327411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:50.327376    2567 generic.go:356] "Generic (PLEG): container finished" podID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerID="2ca5bde32be8d76191f4c6ece5b2299799cae1247d7c6565a0e3d9eb9e9fa7d8" exitCode=6
Jul 18 01:11:50.327782 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:50.327447    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" event={"ID":"839f2c34-4010-4831-96d4-f6b04f09e5ed","Type":"ContainerDied","Data":"2ca5bde32be8d76191f4c6ece5b2299799cae1247d7c6565a0e3d9eb9e9fa7d8"}
Jul 18 01:11:50.327782 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:50.327511    2567 scope.go:117] "RemoveContainer" containerID="ca56ffc4b8ad48b3a46cafbfc3dbe1d0bfe2a2c47656e9b179b11dc6c33d9d9e"
Jul 18 01:11:50.327904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:50.327814    2567 scope.go:117] "RemoveContainer" containerID="2ca5bde32be8d76191f4c6ece5b2299799cae1247d7c6565a0e3d9eb9e9fa7d8"
Jul 18 01:11:50.328083 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:11:50.328062    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6_odh-ai-gateway-infra(839f2c34-4010-4831-96d4-f6b04f09e5ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed"
Jul 18 01:11:50.349162 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:50.349129    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"]
Jul 18 01:11:51.463786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:51.463755    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:51.471107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:51.471084    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:51.528017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:51.527984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:11:51.582278 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:51.582242    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839f2c34-4010-4831-96d4-f6b04f09e5ed-maas-api-service-ca\") pod \"839f2c34-4010-4831-96d4-f6b04f09e5ed\" (UID: \"839f2c34-4010-4831-96d4-f6b04f09e5ed\") "
Jul 18 01:11:51.582656 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:51.582631    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/839f2c34-4010-4831-96d4-f6b04f09e5ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "839f2c34-4010-4831-96d4-f6b04f09e5ed" (UID: "839f2c34-4010-4831-96d4-f6b04f09e5ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:11:51.683634 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:51.683542    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839f2c34-4010-4831-96d4-f6b04f09e5ed-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:11:52.339734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:52.339701    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"
Jul 18 01:11:52.339734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:52.339709    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6" event={"ID":"839f2c34-4010-4831-96d4-f6b04f09e5ed","Type":"ContainerDied","Data":"7c9cf6cdb513cc49851576d1c187e6d1406db2c12cb07195c86e2195703e98bb"}
Jul 18 01:11:52.340047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:52.339776    2567 scope.go:117] "RemoveContainer" containerID="2ca5bde32be8d76191f4c6ece5b2299799cae1247d7c6565a0e3d9eb9e9fa7d8"
Jul 18 01:11:52.364190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:52.364161    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"]
Jul 18 01:11:52.368508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:52.368479    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-wd4r6"]
Jul 18 01:11:52.918970 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:11:52.918935    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" path="/var/lib/kubelet/pods/839f2c34-4010-4831-96d4-f6b04f09e5ed/volumes"
Jul 18 01:12:10.467770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.467732    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"]
Jul 18 01:12:10.468149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468090    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.468149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468101    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.468149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468116    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.468149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468122    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.468149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468137    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:12:10.468149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468143    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8d1595-159a-4dc4-8b4f-359e24a4d2ca" containerName="revoke-keys"
Jul 18 01:12:10.468373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468204    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.468373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468212    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.468373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.468218    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:10.472593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.472573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:10.474933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.474906    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:12:10.475127 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.475109    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:12:10.478608 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.478584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"]
Jul 18 01:12:10.540277 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.540240    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"]
Jul 18 01:12:10.540613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.540564    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy" containerID="cri-o://c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18" gracePeriod=30
Jul 18 01:12:10.552942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.552915    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04185bd1-6c22-429f-95d3-5a15babb15f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84\" (UID: \"04185bd1-6c22-429f-95d3-5a15babb15f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:10.570286 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.570246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:10.653737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.653699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04185bd1-6c22-429f-95d3-5a15babb15f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84\" (UID: \"04185bd1-6c22-429f-95d3-5a15babb15f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:10.654415 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.654396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04185bd1-6c22-429f-95d3-5a15babb15f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84\" (UID: \"04185bd1-6c22-429f-95d3-5a15babb15f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:10.665419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.665381    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:10.785342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.785221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:10.931383 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:10.931347    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"]
Jul 18 01:12:11.442887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:11.442802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerStarted","Data":"86460d4941f2e5d036b0ec03fcb52e239ec94cd980f8baa9e92e8c9e648acf48"}
Jul 18 01:12:11.442887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:11.442858    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerStarted","Data":"0d472d0a0d2d830d080e9dbf2358b3ec5d1446f2cb10314c812c1b104d2aae35"}
Jul 18 01:12:11.457841 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:11.457788    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" podStartSLOduration=1.457768854 podStartE2EDuration="1.457768854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:12:11.456938858 +0000 UTC m=+1910.984443820" watchObservedRunningTime="2026-07-18 01:12:11.457768854 +0000 UTC m=+1910.985273815"
Jul 18 01:12:12.450282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:12.450232    2567 generic.go:356] "Generic (PLEG): container finished" podID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerID="86460d4941f2e5d036b0ec03fcb52e239ec94cd980f8baa9e92e8c9e648acf48" exitCode=7
Jul 18 01:12:12.450666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:12.450365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerDied","Data":"86460d4941f2e5d036b0ec03fcb52e239ec94cd980f8baa9e92e8c9e648acf48"}
Jul 18 01:12:12.450666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:12.450524    2567 scope.go:117] "RemoveContainer" containerID="86460d4941f2e5d036b0ec03fcb52e239ec94cd980f8baa9e92e8c9e648acf48"
Jul 18 01:12:13.459021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:13.458921    2567 generic.go:356] "Generic (PLEG): container finished" podID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerID="2055ab39e6343cff7fe44d32c6a03f5e66d4184839172a33194808a36246e982" exitCode=7
Jul 18 01:12:13.459021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:13.458963    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerDied","Data":"2055ab39e6343cff7fe44d32c6a03f5e66d4184839172a33194808a36246e982"}
Jul 18 01:12:13.459553 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:13.459044    2567 scope.go:117] "RemoveContainer" containerID="86460d4941f2e5d036b0ec03fcb52e239ec94cd980f8baa9e92e8c9e648acf48"
Jul 18 01:12:13.459553 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:13.459326    2567 scope.go:117] "RemoveContainer" containerID="2055ab39e6343cff7fe44d32c6a03f5e66d4184839172a33194808a36246e982"
Jul 18 01:12:13.459642 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:13.459566    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84_odh-ai-gateway-infra(04185bd1-6c22-429f-95d3-5a15babb15f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0"
Jul 18 01:12:13.808820 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:13.808784    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:12:14.465596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:14.465561    2567 scope.go:117] "RemoveContainer" containerID="2055ab39e6343cff7fe44d32c6a03f5e66d4184839172a33194808a36246e982"
Jul 18 01:12:14.466087 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:14.465804    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84_odh-ai-gateway-infra(04185bd1-6c22-429f-95d3-5a15babb15f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0"
Jul 18 01:12:15.801878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.801848    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:12:15.903278 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903227    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf99p\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-kube-api-access-hf99p\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903278 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903278    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-podinfo\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903305    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-data\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903323    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-credential-socket\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903393    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-token\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903433    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-socket\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903472    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-envoy\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903494    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istiod-ca-cert\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903554    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-certs\") pod \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\" (UID: \"6d248a30-f020-4dfd-9ae4-d15eaa834ea7\") "
Jul 18 01:12:15.903833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903701    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:12:15.903946 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903879    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:15.903946 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903906    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:12:15.904168 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.903992    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:12:15.904168 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.904167    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:12:15.904452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.904429    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:12:15.905815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.905788    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:12:15.905815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.905788    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:12:15.905964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.905815    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-kube-api-access-hf99p" (OuterVolumeSpecName: "kube-api-access-hf99p") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "kube-api-access-hf99p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:12:15.906027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:15.906009    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d248a30-f020-4dfd-9ae4-d15eaa834ea7" (UID: "6d248a30-f020-4dfd-9ae4-d15eaa834ea7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:12:16.006759 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006720    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.006759 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006757    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.007013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006771    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.007013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006784    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.007013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006795    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.007013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006806    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.007013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006817    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf99p\" (UniqueName: \"kubernetes.io/projected/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-kube-api-access-hf99p\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.007013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.006831    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d248a30-f020-4dfd-9ae4-d15eaa834ea7-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:16.036167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.036135    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:16.433005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.432976    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:16.477575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.477543    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerID="c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18" exitCode=0
Jul 18 01:12:16.477769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.477609    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"
Jul 18 01:12:16.477769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.477632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" event={"ID":"6d248a30-f020-4dfd-9ae4-d15eaa834ea7","Type":"ContainerDied","Data":"c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18"}
Jul 18 01:12:16.477769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.477711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr" event={"ID":"6d248a30-f020-4dfd-9ae4-d15eaa834ea7","Type":"ContainerDied","Data":"b86ffb1a5a1e33dcd4c4ce33722dfc97f60b5d8c9b718c396fa32200c8abc4ea"}
Jul 18 01:12:16.477769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.477735    2567 scope.go:117] "RemoveContainer" containerID="c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18"
Jul 18 01:12:16.487765 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.487746    2567 scope.go:117] "RemoveContainer" containerID="ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"
Jul 18 01:12:16.497109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.497087    2567 scope.go:117] "RemoveContainer" containerID="c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18"
Jul 18 01:12:16.497462 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:16.497433    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18\": container with ID starting with c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18 not found: ID does not exist" containerID="c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18"
Jul 18 01:12:16.497567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.497471    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18"} err="failed to get container status \"c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18\": rpc error: code = NotFound desc = could not find container \"c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18\": container with ID starting with c7a8a00a42d6955eb5abe30950bb336c10a260dd9a460aaa290999cc53e63b18 not found: ID does not exist"
Jul 18 01:12:16.497567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.497499    2567 scope.go:117] "RemoveContainer" containerID="ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"
Jul 18 01:12:16.497809 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:16.497792    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4\": container with ID starting with ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4 not found: ID does not exist" containerID="ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"
Jul 18 01:12:16.497847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.497815    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4"} err="failed to get container status \"ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4\": rpc error: code = NotFound desc = could not find container \"ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4\": container with ID starting with ea1951dcd402e9a9113e11bfaca8f7c8e729761bf35271aa97240d01d57081f4 not found: ID does not exist"
Jul 18 01:12:16.498732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.498713    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"]
Jul 18 01:12:16.502055 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.502031    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1b6716ba-gw-openshift-default-74f64ffc95-rs9qr"]
Jul 18 01:12:16.919290 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:16.919237    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" path="/var/lib/kubelet/pods/6d248a30-f020-4dfd-9ae4-d15eaa834ea7/volumes"
Jul 18 01:12:17.122737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:17.122705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:21.955245 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:21.955210    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:22.144471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.144433    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"]
Jul 18 01:12:22.145023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.145003    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy"
Jul 18 01:12:22.145077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.145030    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy"
Jul 18 01:12:22.145077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.145044    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:22.145077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.145053    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="839f2c34-4010-4831-96d4-f6b04f09e5ed" containerName="revoke-keys"
Jul 18 01:12:22.145192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.145180    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy"
Jul 18 01:12:22.148702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.148681    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.151441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.151412    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d5466ad4-openshift-default-dockercfg-kk95f\""
Jul 18 01:12:22.162385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.162357    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"]
Jul 18 01:12:22.266307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-podinfo\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266234    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fq55\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-kube-api-access-9fq55\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266261    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-data\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3deed382-f91d-43d3-a8f6-982fd5933e71-istiod-ca-cert\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-certs\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266406    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-credential-socket\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-socket\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-envoy\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.266795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.266591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-token\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.367658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-credential-socket\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.367843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367673    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-socket\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.367843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-envoy\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.367843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367772    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-token\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.367843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-podinfo\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.367843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367834    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fq55\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-kube-api-access-9fq55\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-data\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3deed382-f91d-43d3-a8f6-982fd5933e71-istiod-ca-cert\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.367949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-certs\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.368134    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-socket\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.368253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-data\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.368345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-certs\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.368352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-credential-socket\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.368975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.368950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3deed382-f91d-43d3-a8f6-982fd5933e71-istiod-ca-cert\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.370231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.370209    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-envoy\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.370528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.370510    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-podinfo\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.376035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.376003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-token\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.376154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.376135    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fq55\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-kube-api-access-9fq55\") pod \"e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") " pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.463535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.463491    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:22.645665 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:12:22.645634    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3deed382_f91d_43d3_a8f6_982fd5933e71.slice/crio-7b7989f98d4b613c7cce5e52fcddcf718f5cc576d99c3c2a9a93ab5ee513af42 WatchSource:0}: Error finding container 7b7989f98d4b613c7cce5e52fcddcf718f5cc576d99c3c2a9a93ab5ee513af42: Status 404 returned error can't find the container with id 7b7989f98d4b613c7cce5e52fcddcf718f5cc576d99c3c2a9a93ab5ee513af42
Jul 18 01:12:22.648369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.648332    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:12:22.648491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.648405    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:12:22.648491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.648437    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:12:22.649047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.649028    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"]
Jul 18 01:12:22.954896 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.954863    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"]
Jul 18 01:12:22.955264 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.955250    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy"
Jul 18 01:12:22.955264 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.955280    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy"
Jul 18 01:12:22.955623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.955379    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d248a30-f020-4dfd-9ae4-d15eaa834ea7" containerName="istio-proxy"
Jul 18 01:12:22.958547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.958528    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:22.965035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:22.965001    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"]
Jul 18 01:12:23.073685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.073651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2226648-709f-485f-9e84-5d61490d46e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp\" (UID: \"f2226648-709f-485f-9e84-5d61490d46e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:23.134417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.134361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:23.174632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.174591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2226648-709f-485f-9e84-5d61490d46e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp\" (UID: \"f2226648-709f-485f-9e84-5d61490d46e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:23.175310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.175258    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2226648-709f-485f-9e84-5d61490d46e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp\" (UID: \"f2226648-709f-485f-9e84-5d61490d46e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:23.271758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.271676    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:23.283876 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.283846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:23.381594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.381560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:23.523482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.523383    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" event={"ID":"3deed382-f91d-43d3-a8f6-982fd5933e71","Type":"ContainerStarted","Data":"fc1fa3fa64a05955ea75756ca591f83da6fb8a3c6e9683659022e30ec009024d"}
Jul 18 01:12:23.523482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.523447    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" event={"ID":"3deed382-f91d-43d3-a8f6-982fd5933e71","Type":"ContainerStarted","Data":"7b7989f98d4b613c7cce5e52fcddcf718f5cc576d99c3c2a9a93ab5ee513af42"}
Jul 18 01:12:23.545981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.545921    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" podStartSLOduration=1.545905264 podStartE2EDuration="1.545905264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:12:23.544218761 +0000 UTC m=+1923.071723723" watchObservedRunningTime="2026-07-18 01:12:23.545905264 +0000 UTC m=+1923.073410223"
Jul 18 01:12:23.548205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.548176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:23.628017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.627993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"]
Jul 18 01:12:23.630332 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:12:23.630292    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2226648_709f_485f_9e84_5d61490d46e0.slice/crio-81ece738ddb4e4d97396ed59119704921d11ed8a1c0f2bc907e6f1a1fee8353c WatchSource:0}: Error finding container 81ece738ddb4e4d97396ed59119704921d11ed8a1c0f2bc907e6f1a1fee8353c: Status 404 returned error can't find the container with id 81ece738ddb4e4d97396ed59119704921d11ed8a1c0f2bc907e6f1a1fee8353c
Jul 18 01:12:23.739504 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:23.739471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:24.000129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.000094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:24.106629 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.106601    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:24.464523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.464484    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:24.469856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.469827    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:24.529696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.529663    2567 generic.go:356] "Generic (PLEG): container finished" podID="f2226648-709f-485f-9e84-5d61490d46e0" containerID="96b1e359373e13dac35e57e54f5ee462b6ee4c2702fb0c5cb48a1db48da5a67b" exitCode=6
Jul 18 01:12:24.529890 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.529745    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" event={"ID":"f2226648-709f-485f-9e84-5d61490d46e0","Type":"ContainerDied","Data":"96b1e359373e13dac35e57e54f5ee462b6ee4c2702fb0c5cb48a1db48da5a67b"}
Jul 18 01:12:24.529890 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.529804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" event={"ID":"f2226648-709f-485f-9e84-5d61490d46e0","Type":"ContainerStarted","Data":"81ece738ddb4e4d97396ed59119704921d11ed8a1c0f2bc907e6f1a1fee8353c"}
Jul 18 01:12:24.530016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.529898    2567 scope.go:117] "RemoveContainer" containerID="96b1e359373e13dac35e57e54f5ee462b6ee4c2702fb0c5cb48a1db48da5a67b"
Jul 18 01:12:24.530190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.530164    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:24.531208 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:24.531193    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:25.474579 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:25.474549    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:12:25.537191 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:25.537148    2567 generic.go:356] "Generic (PLEG): container finished" podID="f2226648-709f-485f-9e84-5d61490d46e0" containerID="8821586a87b256fbb75109c8f6476a0c22000ad2f17cd7338329287a50bc0bad" exitCode=6
Jul 18 01:12:25.537424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:25.537241    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" event={"ID":"f2226648-709f-485f-9e84-5d61490d46e0","Type":"ContainerDied","Data":"8821586a87b256fbb75109c8f6476a0c22000ad2f17cd7338329287a50bc0bad"}
Jul 18 01:12:25.537424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:25.537332    2567 scope.go:117] "RemoveContainer" containerID="96b1e359373e13dac35e57e54f5ee462b6ee4c2702fb0c5cb48a1db48da5a67b"
Jul 18 01:12:25.537562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:25.537487    2567 scope.go:117] "RemoveContainer" containerID="8821586a87b256fbb75109c8f6476a0c22000ad2f17cd7338329287a50bc0bad"
Jul 18 01:12:25.537742 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:25.537719    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp_odh-ai-gateway-infra(f2226648-709f-485f-9e84-5d61490d46e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" podUID="f2226648-709f-485f-9e84-5d61490d46e0"
Jul 18 01:12:26.544526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:26.544496    2567 scope.go:117] "RemoveContainer" containerID="8821586a87b256fbb75109c8f6476a0c22000ad2f17cd7338329287a50bc0bad"
Jul 18 01:12:26.545012 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:26.544744    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp_odh-ai-gateway-infra(f2226648-709f-485f-9e84-5d61490d46e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" podUID="f2226648-709f-485f-9e84-5d61490d46e0"
Jul 18 01:12:26.913872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:26.913835    2567 scope.go:117] "RemoveContainer" containerID="2055ab39e6343cff7fe44d32c6a03f5e66d4184839172a33194808a36246e982"
Jul 18 01:12:26.997161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:26.997123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:12:27.551986 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:27.551893    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerStarted","Data":"2c0b0ea7fe8ed2b771bc201964ccf27c0d9718114baa06494282c12e17c7e77b"}
Jul 18 01:12:28.360126 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.360085    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"]
Jul 18 01:12:28.364755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.364713    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:28.373972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.373931    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"]
Jul 18 01:12:28.525703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.525656    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4494a689-30bb-4cd4-9bd8-d10cdd009414-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh\" (UID: \"4494a689-30bb-4cd4-9bd8-d10cdd009414\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:28.563525 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.563481    2567 generic.go:356] "Generic (PLEG): container finished" podID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerID="2c0b0ea7fe8ed2b771bc201964ccf27c0d9718114baa06494282c12e17c7e77b" exitCode=6
Jul 18 01:12:28.564028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.563551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerDied","Data":"2c0b0ea7fe8ed2b771bc201964ccf27c0d9718114baa06494282c12e17c7e77b"}
Jul 18 01:12:28.564028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.563629    2567 scope.go:117] "RemoveContainer" containerID="2055ab39e6343cff7fe44d32c6a03f5e66d4184839172a33194808a36246e982"
Jul 18 01:12:28.564150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.564092    2567 scope.go:117] "RemoveContainer" containerID="2c0b0ea7fe8ed2b771bc201964ccf27c0d9718114baa06494282c12e17c7e77b"
Jul 18 01:12:28.564469 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:28.564435    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84_odh-ai-gateway-infra(04185bd1-6c22-429f-95d3-5a15babb15f0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0"
Jul 18 01:12:28.580304 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.578500    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"]
Jul 18 01:12:28.626644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.626556    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4494a689-30bb-4cd4-9bd8-d10cdd009414-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh\" (UID: \"4494a689-30bb-4cd4-9bd8-d10cdd009414\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:28.627175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.627157    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4494a689-30bb-4cd4-9bd8-d10cdd009414-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh\" (UID: \"4494a689-30bb-4cd4-9bd8-d10cdd009414\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:28.686978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.686940    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:28.878331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:28.877839    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"]
Jul 18 01:12:28.880459 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:12:28.880423    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4494a689_30bb_4cd4_9bd8_d10cdd009414.slice/crio-ad479d2b5bb1d20f9ea648d193a034215fc1fb9c286030de440d4f972ba8427e WatchSource:0}: Error finding container ad479d2b5bb1d20f9ea648d193a034215fc1fb9c286030de440d4f972ba8427e: Status 404 returned error can't find the container with id ad479d2b5bb1d20f9ea648d193a034215fc1fb9c286030de440d4f972ba8427e
Jul 18 01:12:29.577408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.577359    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" event={"ID":"4494a689-30bb-4cd4-9bd8-d10cdd009414","Type":"ContainerStarted","Data":"74b26bf5800748532e987b0b948e2c4e9590de7104b33cd0cb68db9d3e013141"}
Jul 18 01:12:29.577961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.577474    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" event={"ID":"4494a689-30bb-4cd4-9bd8-d10cdd009414","Type":"ContainerStarted","Data":"ad479d2b5bb1d20f9ea648d193a034215fc1fb9c286030de440d4f972ba8427e"}
Jul 18 01:12:29.596422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.596355    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" podStartSLOduration=1.5963321879999999 podStartE2EDuration="1.596332188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:12:29.593426687 +0000 UTC m=+1929.120931647" watchObservedRunningTime="2026-07-18 01:12:29.596332188 +0000 UTC m=+1929.123837151"
Jul 18 01:12:29.740434 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.740411    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:29.838812 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.838719    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04185bd1-6c22-429f-95d3-5a15babb15f0-maas-api-service-ca\") pod \"04185bd1-6c22-429f-95d3-5a15babb15f0\" (UID: \"04185bd1-6c22-429f-95d3-5a15babb15f0\") "
Jul 18 01:12:29.839091 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.839068    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04185bd1-6c22-429f-95d3-5a15babb15f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04185bd1-6c22-429f-95d3-5a15babb15f0" (UID: "04185bd1-6c22-429f-95d3-5a15babb15f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:12:29.940057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:29.940009    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04185bd1-6c22-429f-95d3-5a15babb15f0-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:30.587116 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.587079    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"
Jul 18 01:12:30.587668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.587124    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84" event={"ID":"04185bd1-6c22-429f-95d3-5a15babb15f0","Type":"ContainerDied","Data":"0d472d0a0d2d830d080e9dbf2358b3ec5d1446f2cb10314c812c1b104d2aae35"}
Jul 18 01:12:30.587668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.587203    2567 scope.go:117] "RemoveContainer" containerID="2c0b0ea7fe8ed2b771bc201964ccf27c0d9718114baa06494282c12e17c7e77b"
Jul 18 01:12:30.588761 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.588739    2567 generic.go:356] "Generic (PLEG): container finished" podID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerID="74b26bf5800748532e987b0b948e2c4e9590de7104b33cd0cb68db9d3e013141" exitCode=7
Jul 18 01:12:30.588850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.588825    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" event={"ID":"4494a689-30bb-4cd4-9bd8-d10cdd009414","Type":"ContainerDied","Data":"74b26bf5800748532e987b0b948e2c4e9590de7104b33cd0cb68db9d3e013141"}
Jul 18 01:12:30.588981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.588966    2567 scope.go:117] "RemoveContainer" containerID="74b26bf5800748532e987b0b948e2c4e9590de7104b33cd0cb68db9d3e013141"
Jul 18 01:12:30.619981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.619948    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"]
Jul 18 01:12:30.627462 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.627437    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7pg84"]
Jul 18 01:12:30.919411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:30.919378    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" path="/var/lib/kubelet/pods/04185bd1-6c22-429f-95d3-5a15babb15f0/volumes"
Jul 18 01:12:31.597113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:31.597075    2567 generic.go:356] "Generic (PLEG): container finished" podID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerID="7c1cfb088ea9e1c4abe38c8be491dc2fc6f511d3cabda92546def0c803e2aeeb" exitCode=7
Jul 18 01:12:31.597641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:31.597166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" event={"ID":"4494a689-30bb-4cd4-9bd8-d10cdd009414","Type":"ContainerDied","Data":"7c1cfb088ea9e1c4abe38c8be491dc2fc6f511d3cabda92546def0c803e2aeeb"}
Jul 18 01:12:31.597641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:31.597233    2567 scope.go:117] "RemoveContainer" containerID="74b26bf5800748532e987b0b948e2c4e9590de7104b33cd0cb68db9d3e013141"
Jul 18 01:12:31.597641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:31.597368    2567 scope.go:117] "RemoveContainer" containerID="7c1cfb088ea9e1c4abe38c8be491dc2fc6f511d3cabda92546def0c803e2aeeb"
Jul 18 01:12:31.597641 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:31.597612    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh_odh-ai-gateway-infra(4494a689-30bb-4cd4-9bd8-d10cdd009414)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414"
Jul 18 01:12:32.605419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:32.605394    2567 scope.go:117] "RemoveContainer" containerID="7c1cfb088ea9e1c4abe38c8be491dc2fc6f511d3cabda92546def0c803e2aeeb"
Jul 18 01:12:32.605815 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:32.605647    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh_odh-ai-gateway-infra(4494a689-30bb-4cd4-9bd8-d10cdd009414)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414"
Jul 18 01:12:34.617747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:34.617717    2567 generic.go:356] "Generic (PLEG): container finished" podID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerID="fc1fa3fa64a05955ea75756ca591f83da6fb8a3c6e9683659022e30ec009024d" exitCode=0
Jul 18 01:12:34.618173 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:34.617795    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" event={"ID":"3deed382-f91d-43d3-a8f6-982fd5933e71","Type":"ContainerDied","Data":"fc1fa3fa64a05955ea75756ca591f83da6fb8a3c6e9683659022e30ec009024d"}
Jul 18 01:12:34.618216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:34.618194    2567 scope.go:117] "RemoveContainer" containerID="fc1fa3fa64a05955ea75756ca591f83da6fb8a3c6e9683659022e30ec009024d"
Jul 18 01:12:34.619176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:34.619146    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:12:34.619235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:34.619218    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:12:34.619288 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:34.619253    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:12:35.624871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:35.624836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" event={"ID":"3deed382-f91d-43d3-a8f6-982fd5933e71","Type":"ContainerStarted","Data":"ce5c203368716b6b3d2ac86aa63e0fc8eca34487eb2695c20b45f88677c389fd"}
Jul 18 01:12:36.464122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:36.464080    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:36.465653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:36.465613    2567 patch_prober.go:28] interesting pod/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 18 01:12:36.465831 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:36.465684    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 18 01:12:37.464867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:37.464830    2567 patch_prober.go:28] interesting pod/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 18 01:12:37.465487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:37.465454    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 18 01:12:38.465017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:38.464965    2567 patch_prober.go:28] interesting pod/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 18 01:12:38.465558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:38.465053    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 18 01:12:38.913245 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:38.913210    2567 scope.go:117] "RemoveContainer" containerID="8821586a87b256fbb75109c8f6476a0c22000ad2f17cd7338329287a50bc0bad"
Jul 18 01:12:39.468649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:39.468571    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:39.469002 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:39.468933    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:39.469599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:39.469583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:12:39.647403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:39.647363    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" event={"ID":"f2226648-709f-485f-9e84-5d61490d46e0","Type":"ContainerStarted","Data":"6c422f8867b8666669971046314d56669626c7d01c6ff5c93d89103e1771a4e4"}
Jul 18 01:12:39.664476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:39.664429    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" podStartSLOduration=17.6644129 podStartE2EDuration="17.6644129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:12:39.661940708 +0000 UTC m=+1939.189445669" watchObservedRunningTime="2026-07-18 01:12:39.6644129 +0000 UTC m=+1939.191917859"
Jul 18 01:12:40.654084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:40.654050    2567 generic.go:356] "Generic (PLEG): container finished" podID="f2226648-709f-485f-9e84-5d61490d46e0" containerID="6c422f8867b8666669971046314d56669626c7d01c6ff5c93d89103e1771a4e4" exitCode=6
Jul 18 01:12:40.654570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:40.654132    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" event={"ID":"f2226648-709f-485f-9e84-5d61490d46e0","Type":"ContainerDied","Data":"6c422f8867b8666669971046314d56669626c7d01c6ff5c93d89103e1771a4e4"}
Jul 18 01:12:40.654570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:40.654202    2567 scope.go:117] "RemoveContainer" containerID="8821586a87b256fbb75109c8f6476a0c22000ad2f17cd7338329287a50bc0bad"
Jul 18 01:12:40.654702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:40.654613    2567 scope.go:117] "RemoveContainer" containerID="6c422f8867b8666669971046314d56669626c7d01c6ff5c93d89103e1771a4e4"
Jul 18 01:12:40.654854 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:40.654825    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp_odh-ai-gateway-infra(f2226648-709f-485f-9e84-5d61490d46e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" podUID="f2226648-709f-485f-9e84-5d61490d46e0"
Jul 18 01:12:40.678704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:40.678674    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"]
Jul 18 01:12:41.802551 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:41.802521    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:41.951027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:41.950927    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2226648-709f-485f-9e84-5d61490d46e0-maas-api-service-ca\") pod \"f2226648-709f-485f-9e84-5d61490d46e0\" (UID: \"f2226648-709f-485f-9e84-5d61490d46e0\") "
Jul 18 01:12:41.951390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:41.951352    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2226648-709f-485f-9e84-5d61490d46e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f2226648-709f-485f-9e84-5d61490d46e0" (UID: "f2226648-709f-485f-9e84-5d61490d46e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:12:42.052097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.052060    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2226648-709f-485f-9e84-5d61490d46e0-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:42.667590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.667567    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"
Jul 18 01:12:42.667781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.667567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp" event={"ID":"f2226648-709f-485f-9e84-5d61490d46e0","Type":"ContainerDied","Data":"81ece738ddb4e4d97396ed59119704921d11ed8a1c0f2bc907e6f1a1fee8353c"}
Jul 18 01:12:42.667781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.667705    2567 scope.go:117] "RemoveContainer" containerID="6c422f8867b8666669971046314d56669626c7d01c6ff5c93d89103e1771a4e4"
Jul 18 01:12:42.687453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.687426    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"]
Jul 18 01:12:42.694956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.694928    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-mp6xp"]
Jul 18 01:12:42.919437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:42.919359    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2226648-709f-485f-9e84-5d61490d46e0" path="/var/lib/kubelet/pods/f2226648-709f-485f-9e84-5d61490d46e0/volumes"
Jul 18 01:12:44.913595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:44.913560    2567 scope.go:117] "RemoveContainer" containerID="7c1cfb088ea9e1c4abe38c8be491dc2fc6f511d3cabda92546def0c803e2aeeb"
Jul 18 01:12:45.684648 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:45.684611    2567 generic.go:356] "Generic (PLEG): container finished" podID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerID="d5a968500aeeb5868023cdd61e374f2d207625db793f4d01775e8ad51f4fad15" exitCode=7
Jul 18 01:12:45.684806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:45.684660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" event={"ID":"4494a689-30bb-4cd4-9bd8-d10cdd009414","Type":"ContainerDied","Data":"d5a968500aeeb5868023cdd61e374f2d207625db793f4d01775e8ad51f4fad15"}
Jul 18 01:12:45.684806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:45.684720    2567 scope.go:117] "RemoveContainer" containerID="7c1cfb088ea9e1c4abe38c8be491dc2fc6f511d3cabda92546def0c803e2aeeb"
Jul 18 01:12:45.685152 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:45.685084    2567 scope.go:117] "RemoveContainer" containerID="d5a968500aeeb5868023cdd61e374f2d207625db793f4d01775e8ad51f4fad15"
Jul 18 01:12:45.685341 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:12:45.685322    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh_odh-ai-gateway-infra(4494a689-30bb-4cd4-9bd8-d10cdd009414)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414"
Jul 18 01:12:46.710538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:46.710508    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"]
Jul 18 01:12:46.865928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:46.865903    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:46.891795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:46.891768    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4494a689-30bb-4cd4-9bd8-d10cdd009414-maas-api-service-ca\") pod \"4494a689-30bb-4cd4-9bd8-d10cdd009414\" (UID: \"4494a689-30bb-4cd4-9bd8-d10cdd009414\") "
Jul 18 01:12:46.892140 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:46.892115    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4494a689-30bb-4cd4-9bd8-d10cdd009414-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4494a689-30bb-4cd4-9bd8-d10cdd009414" (UID: "4494a689-30bb-4cd4-9bd8-d10cdd009414"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:12:46.992946 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:46.992843    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4494a689-30bb-4cd4-9bd8-d10cdd009414-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:12:47.697183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:47.697154    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"
Jul 18 01:12:47.697387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:47.697144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh" event={"ID":"4494a689-30bb-4cd4-9bd8-d10cdd009414","Type":"ContainerDied","Data":"ad479d2b5bb1d20f9ea648d193a034215fc1fb9c286030de440d4f972ba8427e"}
Jul 18 01:12:47.697387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:47.697316    2567 scope.go:117] "RemoveContainer" containerID="d5a968500aeeb5868023cdd61e374f2d207625db793f4d01775e8ad51f4fad15"
Jul 18 01:12:47.712376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:47.712346    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"]
Jul 18 01:12:47.715767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:47.715743    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-hfvfh"]
Jul 18 01:12:48.919626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:12:48.919592    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" path="/var/lib/kubelet/pods/4494a689-30bb-4cd4-9bd8-d10cdd009414/volumes"
Jul 18 01:13:10.511832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.511795    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"]
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512164    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512175    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512188    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512193    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512200    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512206    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512215    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512220    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512235    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512240    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512248    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512253    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512323    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512332    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512338    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512344    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512350    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512356    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512361    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:10.512717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.512367    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:10.515627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.515609    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:10.518129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.518099    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:13:10.518365 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.518177    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:13:10.521966 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.521942    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"]
Jul 18 01:13:10.604314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.604252    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3283e0-1969-493a-ab00-03c7f4f1b979-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj\" (UID: \"0e3283e0-1969-493a-ab00-03c7f4f1b979\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:10.705110 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.705071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3283e0-1969-493a-ab00-03c7f4f1b979-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj\" (UID: \"0e3283e0-1969-493a-ab00-03c7f4f1b979\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:10.705773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.705740    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3283e0-1969-493a-ab00-03c7f4f1b979-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj\" (UID: \"0e3283e0-1969-493a-ab00-03c7f4f1b979\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:10.829052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.829017    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:10.967984 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:10.967910    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"]
Jul 18 01:13:10.970527 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:13:10.970495    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3283e0_1969_493a_ab00_03c7f4f1b979.slice/crio-8ee4931b54a6a44ad9703ddb07c69f747b308f2e5c77aa3086b87f3ebcb965a8 WatchSource:0}: Error finding container 8ee4931b54a6a44ad9703ddb07c69f747b308f2e5c77aa3086b87f3ebcb965a8: Status 404 returned error can't find the container with id 8ee4931b54a6a44ad9703ddb07c69f747b308f2e5c77aa3086b87f3ebcb965a8
Jul 18 01:13:11.818749 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:11.818659    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerID="280081c0479baaa4af610a2b2564be32797aefff3381b97485d48d3e83ef3c16" exitCode=6
Jul 18 01:13:11.819094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:11.818749    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" event={"ID":"0e3283e0-1969-493a-ab00-03c7f4f1b979","Type":"ContainerDied","Data":"280081c0479baaa4af610a2b2564be32797aefff3381b97485d48d3e83ef3c16"}
Jul 18 01:13:11.819094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:11.818809    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" event={"ID":"0e3283e0-1969-493a-ab00-03c7f4f1b979","Type":"ContainerStarted","Data":"8ee4931b54a6a44ad9703ddb07c69f747b308f2e5c77aa3086b87f3ebcb965a8"}
Jul 18 01:13:11.819094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:11.818941    2567 scope.go:117] "RemoveContainer" containerID="280081c0479baaa4af610a2b2564be32797aefff3381b97485d48d3e83ef3c16"
Jul 18 01:13:12.825632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.825593    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerID="b6965adbb1d60c9ddd1e0dd992bfed8117a5708d7a65d730e350c4cf710d362f" exitCode=6
Jul 18 01:13:12.826016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.825668    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" event={"ID":"0e3283e0-1969-493a-ab00-03c7f4f1b979","Type":"ContainerDied","Data":"b6965adbb1d60c9ddd1e0dd992bfed8117a5708d7a65d730e350c4cf710d362f"}
Jul 18 01:13:12.826016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.825800    2567 scope.go:117] "RemoveContainer" containerID="280081c0479baaa4af610a2b2564be32797aefff3381b97485d48d3e83ef3c16"
Jul 18 01:13:12.826137 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.826020    2567 scope.go:117] "RemoveContainer" containerID="b6965adbb1d60c9ddd1e0dd992bfed8117a5708d7a65d730e350c4cf710d362f"
Jul 18 01:13:12.826249 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:12.826230    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj_odh-ai-gateway-infra(0e3283e0-1969-493a-ab00-03c7f4f1b979)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979"
Jul 18 01:13:12.996608 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.996564    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"]
Jul 18 01:13:12.997045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997026    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:12.997146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997046    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="04185bd1-6c22-429f-95d3-5a15babb15f0" containerName="revoke-keys"
Jul 18 01:13:12.997146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997076    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:12.997146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997084    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4494a689-30bb-4cd4-9bd8-d10cdd009414" containerName="revoke-keys"
Jul 18 01:13:12.997146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997101    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:12.997146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997108    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:12.997471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:12.997201    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2226648-709f-485f-9e84-5d61490d46e0" containerName="revoke-keys"
Jul 18 01:13:13.001492 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.001470    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:13.007404 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.007376    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"]
Jul 18 01:13:13.125613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.125505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26d2fd65-06d6-408c-9ab1-fd21cb145e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj\" (UID: \"26d2fd65-06d6-408c-9ab1-fd21cb145e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:13.226769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.226726    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26d2fd65-06d6-408c-9ab1-fd21cb145e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj\" (UID: \"26d2fd65-06d6-408c-9ab1-fd21cb145e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:13.227395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.227374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26d2fd65-06d6-408c-9ab1-fd21cb145e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj\" (UID: \"26d2fd65-06d6-408c-9ab1-fd21cb145e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:13.314070 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.314033    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:13.453887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.453862    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"]
Jul 18 01:13:13.456756 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:13:13.456725    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d2fd65_06d6_408c_9ab1_fd21cb145e0e.slice/crio-d40e75524b7b68737e20fde4d3e34e5c536ae12babc89738f36843f5e631a3f1 WatchSource:0}: Error finding container d40e75524b7b68737e20fde4d3e34e5c536ae12babc89738f36843f5e631a3f1: Status 404 returned error can't find the container with id d40e75524b7b68737e20fde4d3e34e5c536ae12babc89738f36843f5e631a3f1
Jul 18 01:13:13.832747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.832717    2567 scope.go:117] "RemoveContainer" containerID="b6965adbb1d60c9ddd1e0dd992bfed8117a5708d7a65d730e350c4cf710d362f"
Jul 18 01:13:13.833184 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:13.832936    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj_odh-ai-gateway-infra(0e3283e0-1969-493a-ab00-03c7f4f1b979)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979"
Jul 18 01:13:13.834215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.834191    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" event={"ID":"26d2fd65-06d6-408c-9ab1-fd21cb145e0e","Type":"ContainerStarted","Data":"aa398a3c9a318c4e7f430158d951742e7168cdb365a9c1182a7581fe0e823a22"}
Jul 18 01:13:13.834373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.834232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" event={"ID":"26d2fd65-06d6-408c-9ab1-fd21cb145e0e","Type":"ContainerStarted","Data":"d40e75524b7b68737e20fde4d3e34e5c536ae12babc89738f36843f5e631a3f1"}
Jul 18 01:13:13.861673 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:13.861618    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" podStartSLOduration=1.8616006729999999 podStartE2EDuration="1.861600673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:13:13.860374692 +0000 UTC m=+1973.387879653" watchObservedRunningTime="2026-07-18 01:13:13.861600673 +0000 UTC m=+1973.389105634"
Jul 18 01:13:14.840547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:14.840512    2567 generic.go:356] "Generic (PLEG): container finished" podID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerID="aa398a3c9a318c4e7f430158d951742e7168cdb365a9c1182a7581fe0e823a22" exitCode=6
Jul 18 01:13:14.840948 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:14.840598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" event={"ID":"26d2fd65-06d6-408c-9ab1-fd21cb145e0e","Type":"ContainerDied","Data":"aa398a3c9a318c4e7f430158d951742e7168cdb365a9c1182a7581fe0e823a22"}
Jul 18 01:13:14.840948 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:14.840772    2567 scope.go:117] "RemoveContainer" containerID="aa398a3c9a318c4e7f430158d951742e7168cdb365a9c1182a7581fe0e823a22"
Jul 18 01:13:15.847156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:15.847120    2567 generic.go:356] "Generic (PLEG): container finished" podID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerID="7dea92b4a704f03a7484ace27303adba01226c84ac27a61100bd843349d788d3" exitCode=6
Jul 18 01:13:15.847779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:15.847157    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" event={"ID":"26d2fd65-06d6-408c-9ab1-fd21cb145e0e","Type":"ContainerDied","Data":"7dea92b4a704f03a7484ace27303adba01226c84ac27a61100bd843349d788d3"}
Jul 18 01:13:15.847779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:15.847225    2567 scope.go:117] "RemoveContainer" containerID="aa398a3c9a318c4e7f430158d951742e7168cdb365a9c1182a7581fe0e823a22"
Jul 18 01:13:15.847779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:15.847538    2567 scope.go:117] "RemoveContainer" containerID="7dea92b4a704f03a7484ace27303adba01226c84ac27a61100bd843349d788d3"
Jul 18 01:13:15.847960 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:15.847790    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj_odh-ai-gateway-infra(26d2fd65-06d6-408c-9ab1-fd21cb145e0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e"
Jul 18 01:13:16.856217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:16.856184    2567 scope.go:117] "RemoveContainer" containerID="7dea92b4a704f03a7484ace27303adba01226c84ac27a61100bd843349d788d3"
Jul 18 01:13:16.856660 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:16.856470    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj_odh-ai-gateway-infra(26d2fd65-06d6-408c-9ab1-fd21cb145e0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e"
Jul 18 01:13:18.401721 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.401686    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"]
Jul 18 01:13:18.406487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.406467    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:18.411878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.411848    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"]
Jul 18 01:13:18.471137 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.471094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d7e4c5e-967f-421c-ac44-3e553301f92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg\" (UID: \"5d7e4c5e-967f-421c-ac44-3e553301f92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:18.572209 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.572166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d7e4c5e-967f-421c-ac44-3e553301f92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg\" (UID: \"5d7e4c5e-967f-421c-ac44-3e553301f92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:18.572851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.572829    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d7e4c5e-967f-421c-ac44-3e553301f92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg\" (UID: \"5d7e4c5e-967f-421c-ac44-3e553301f92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:18.718183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.718095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:18.857840 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:18.857816    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"]
Jul 18 01:13:18.860768 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:13:18.860738    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7e4c5e_967f_421c_ac44_3e553301f92b.slice/crio-d740723254eebcfa127947e7fcb7b2485eddf9c9ae37a160b6e49b30ead1d414 WatchSource:0}: Error finding container d740723254eebcfa127947e7fcb7b2485eddf9c9ae37a160b6e49b30ead1d414: Status 404 returned error can't find the container with id d740723254eebcfa127947e7fcb7b2485eddf9c9ae37a160b6e49b30ead1d414
Jul 18 01:13:19.872736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:19.872704    2567 generic.go:356] "Generic (PLEG): container finished" podID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerID="d76534e8dc616f5a594d20c7ac64e40b932ad5e04fc9cb83223058039682461c" exitCode=7
Jul 18 01:13:19.873146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:19.872789    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" event={"ID":"5d7e4c5e-967f-421c-ac44-3e553301f92b","Type":"ContainerDied","Data":"d76534e8dc616f5a594d20c7ac64e40b932ad5e04fc9cb83223058039682461c"}
Jul 18 01:13:19.873146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:19.872852    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" event={"ID":"5d7e4c5e-967f-421c-ac44-3e553301f92b","Type":"ContainerStarted","Data":"d740723254eebcfa127947e7fcb7b2485eddf9c9ae37a160b6e49b30ead1d414"}
Jul 18 01:13:19.873146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:19.873004    2567 scope.go:117] "RemoveContainer" containerID="d76534e8dc616f5a594d20c7ac64e40b932ad5e04fc9cb83223058039682461c"
Jul 18 01:13:20.879322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:20.879283    2567 generic.go:356] "Generic (PLEG): container finished" podID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerID="1359bbb9808554bd0dc312e032bb858c49a3be40ca2d52ab45d0e718cd4a51fa" exitCode=7
Jul 18 01:13:20.879730 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:20.879362    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" event={"ID":"5d7e4c5e-967f-421c-ac44-3e553301f92b","Type":"ContainerDied","Data":"1359bbb9808554bd0dc312e032bb858c49a3be40ca2d52ab45d0e718cd4a51fa"}
Jul 18 01:13:20.879730 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:20.879428    2567 scope.go:117] "RemoveContainer" containerID="d76534e8dc616f5a594d20c7ac64e40b932ad5e04fc9cb83223058039682461c"
Jul 18 01:13:20.879730 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:20.879604    2567 scope.go:117] "RemoveContainer" containerID="1359bbb9808554bd0dc312e032bb858c49a3be40ca2d52ab45d0e718cd4a51fa"
Jul 18 01:13:20.879869 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:20.879850    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg_odh-ai-gateway-infra(5d7e4c5e-967f-421c-ac44-3e553301f92b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b"
Jul 18 01:13:21.886490 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:21.886451    2567 scope.go:117] "RemoveContainer" containerID="1359bbb9808554bd0dc312e032bb858c49a3be40ca2d52ab45d0e718cd4a51fa"
Jul 18 01:13:21.886937 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:21.886739    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg_odh-ai-gateway-infra(5d7e4c5e-967f-421c-ac44-3e553301f92b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b"
Jul 18 01:13:22.256077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:22.256000    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:13:22.323248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:22.323204    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:13:28.913645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:28.913612    2567 scope.go:117] "RemoveContainer" containerID="7dea92b4a704f03a7484ace27303adba01226c84ac27a61100bd843349d788d3"
Jul 18 01:13:28.914124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:28.913696    2567 scope.go:117] "RemoveContainer" containerID="b6965adbb1d60c9ddd1e0dd992bfed8117a5708d7a65d730e350c4cf710d362f"
Jul 18 01:13:29.934858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.934771    2567 generic.go:356] "Generic (PLEG): container finished" podID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerID="2abf4711ca90aad1d9694536f13119ad32f972e27dd6d87c85d10f6dc532ec12" exitCode=6
Jul 18 01:13:29.934858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.934843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" event={"ID":"26d2fd65-06d6-408c-9ab1-fd21cb145e0e","Type":"ContainerDied","Data":"2abf4711ca90aad1d9694536f13119ad32f972e27dd6d87c85d10f6dc532ec12"}
Jul 18 01:13:29.935390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.934936    2567 scope.go:117] "RemoveContainer" containerID="7dea92b4a704f03a7484ace27303adba01226c84ac27a61100bd843349d788d3"
Jul 18 01:13:29.935390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.935222    2567 scope.go:117] "RemoveContainer" containerID="2abf4711ca90aad1d9694536f13119ad32f972e27dd6d87c85d10f6dc532ec12"
Jul 18 01:13:29.935504 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:29.935485    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj_odh-ai-gateway-infra(26d2fd65-06d6-408c-9ab1-fd21cb145e0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e"
Jul 18 01:13:29.936902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.936880    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerID="b46f083a6b83df053a41f9e8a4a2539578e439aacc7893fbbe09b9120799f7b5" exitCode=6
Jul 18 01:13:29.937004 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.936929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" event={"ID":"0e3283e0-1969-493a-ab00-03c7f4f1b979","Type":"ContainerDied","Data":"b46f083a6b83df053a41f9e8a4a2539578e439aacc7893fbbe09b9120799f7b5"}
Jul 18 01:13:29.937209 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.937196    2567 scope.go:117] "RemoveContainer" containerID="b46f083a6b83df053a41f9e8a4a2539578e439aacc7893fbbe09b9120799f7b5"
Jul 18 01:13:29.937419 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:29.937402    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj_odh-ai-gateway-infra(0e3283e0-1969-493a-ab00-03c7f4f1b979)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979"
Jul 18 01:13:29.946796 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:29.946778    2567 scope.go:117] "RemoveContainer" containerID="b6965adbb1d60c9ddd1e0dd992bfed8117a5708d7a65d730e350c4cf710d362f"
Jul 18 01:13:30.959144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:30.959106    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"]
Jul 18 01:13:30.971640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:30.971611    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"]
Jul 18 01:13:31.123512 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.123487    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:31.148047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.148021    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:31.287059 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.286945    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3283e0-1969-493a-ab00-03c7f4f1b979-maas-api-service-ca\") pod \"0e3283e0-1969-493a-ab00-03c7f4f1b979\" (UID: \"0e3283e0-1969-493a-ab00-03c7f4f1b979\") "
Jul 18 01:13:31.287059 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.287017    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26d2fd65-06d6-408c-9ab1-fd21cb145e0e-maas-api-service-ca\") pod \"26d2fd65-06d6-408c-9ab1-fd21cb145e0e\" (UID: \"26d2fd65-06d6-408c-9ab1-fd21cb145e0e\") "
Jul 18 01:13:31.287465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.287437    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3283e0-1969-493a-ab00-03c7f4f1b979-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e3283e0-1969-493a-ab00-03c7f4f1b979" (UID: "0e3283e0-1969-493a-ab00-03c7f4f1b979"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:13:31.287555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.287491    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d2fd65-06d6-408c-9ab1-fd21cb145e0e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26d2fd65-06d6-408c-9ab1-fd21cb145e0e" (UID: "26d2fd65-06d6-408c-9ab1-fd21cb145e0e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:13:31.387858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.387820    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3283e0-1969-493a-ab00-03c7f4f1b979-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:13:31.387858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.387853    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26d2fd65-06d6-408c-9ab1-fd21cb145e0e-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:13:31.951071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.951036    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"
Jul 18 01:13:31.951303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.951072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj" event={"ID":"26d2fd65-06d6-408c-9ab1-fd21cb145e0e","Type":"ContainerDied","Data":"d40e75524b7b68737e20fde4d3e34e5c536ae12babc89738f36843f5e631a3f1"}
Jul 18 01:13:31.951303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.951133    2567 scope.go:117] "RemoveContainer" containerID="2abf4711ca90aad1d9694536f13119ad32f972e27dd6d87c85d10f6dc532ec12"
Jul 18 01:13:31.952623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.952592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj" event={"ID":"0e3283e0-1969-493a-ab00-03c7f4f1b979","Type":"ContainerDied","Data":"8ee4931b54a6a44ad9703ddb07c69f747b308f2e5c77aa3086b87f3ebcb965a8"}
Jul 18 01:13:31.952623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.952609    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"
Jul 18 01:13:31.962999 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.962808    2567 scope.go:117] "RemoveContainer" containerID="b46f083a6b83df053a41f9e8a4a2539578e439aacc7893fbbe09b9120799f7b5"
Jul 18 01:13:31.977173 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.977145    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"]
Jul 18 01:13:31.982794 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.982766    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-tqllj"]
Jul 18 01:13:31.993044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.993019    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"]
Jul 18 01:13:31.996373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:31.996344    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-fx2cj"]
Jul 18 01:13:32.920061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:32.920029    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" path="/var/lib/kubelet/pods/0e3283e0-1969-493a-ab00-03c7f4f1b979/volumes"
Jul 18 01:13:32.920429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:32.920415    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" path="/var/lib/kubelet/pods/26d2fd65-06d6-408c-9ab1-fd21cb145e0e/volumes"
Jul 18 01:13:33.913752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:33.913720    2567 scope.go:117] "RemoveContainer" containerID="1359bbb9808554bd0dc312e032bb858c49a3be40ca2d52ab45d0e718cd4a51fa"
Jul 18 01:13:34.970193 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:34.970101    2567 generic.go:356] "Generic (PLEG): container finished" podID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerID="19206386bcefb809f2639685de0b19516a1ceb960f4d136d6c2ba776f476f391" exitCode=7
Jul 18 01:13:34.970193 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:34.970170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" event={"ID":"5d7e4c5e-967f-421c-ac44-3e553301f92b","Type":"ContainerDied","Data":"19206386bcefb809f2639685de0b19516a1ceb960f4d136d6c2ba776f476f391"}
Jul 18 01:13:34.970736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:34.970229    2567 scope.go:117] "RemoveContainer" containerID="1359bbb9808554bd0dc312e032bb858c49a3be40ca2d52ab45d0e718cd4a51fa"
Jul 18 01:13:34.970736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:34.970563    2567 scope.go:117] "RemoveContainer" containerID="19206386bcefb809f2639685de0b19516a1ceb960f4d136d6c2ba776f476f391"
Jul 18 01:13:34.970905 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:34.970812    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg_odh-ai-gateway-infra(5d7e4c5e-967f-421c-ac44-3e553301f92b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b"
Jul 18 01:13:35.994736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:35.994696    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"]
Jul 18 01:13:36.135750 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.135726    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:36.230148 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.230119    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d7e4c5e-967f-421c-ac44-3e553301f92b-maas-api-service-ca\") pod \"5d7e4c5e-967f-421c-ac44-3e553301f92b\" (UID: \"5d7e4c5e-967f-421c-ac44-3e553301f92b\") "
Jul 18 01:13:36.230527 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.230487    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d7e4c5e-967f-421c-ac44-3e553301f92b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d7e4c5e-967f-421c-ac44-3e553301f92b" (UID: "5d7e4c5e-967f-421c-ac44-3e553301f92b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:13:36.331484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.331450    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d7e4c5e-967f-421c-ac44-3e553301f92b-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:13:36.951877 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:13:36.951839    2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7e4c5e_967f_421c_ac44_3e553301f92b.slice/crio-d740723254eebcfa127947e7fcb7b2485eddf9c9ae37a160b6e49b30ead1d414\": RecentStats: unable to find data in memory cache]"
Jul 18 01:13:36.983258 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.983230    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"
Jul 18 01:13:36.983440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.983229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg" event={"ID":"5d7e4c5e-967f-421c-ac44-3e553301f92b","Type":"ContainerDied","Data":"d740723254eebcfa127947e7fcb7b2485eddf9c9ae37a160b6e49b30ead1d414"}
Jul 18 01:13:36.983440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.983368    2567 scope.go:117] "RemoveContainer" containerID="19206386bcefb809f2639685de0b19516a1ceb960f4d136d6c2ba776f476f391"
Jul 18 01:13:36.999084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:36.999053    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"]
Jul 18 01:13:37.003851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:37.003825    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-rc7qg"]
Jul 18 01:13:38.919852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:13:38.919818    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" path="/var/lib/kubelet/pods/5d7e4c5e-967f-421c-ac44-3e553301f92b/volumes"
Jul 18 01:14:03.043218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043181    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"]
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043553    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043564    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043575    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043580    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043588    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043594    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043604    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043610    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043626    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043631    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043638    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043642    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043651    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043656    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.043704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043710    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043718    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043724    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043729    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043736    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043740    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043746    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043752    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="26d2fd65-06d6-408c-9ab1-fd21cb145e0e" containerName="revoke-keys"
Jul 18 01:14:03.044212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.043758    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:03.046799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.046783    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:03.049079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.049055    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:14:03.049197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.049094    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:14:03.052785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.052760    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"]
Jul 18 01:14:03.158717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.158678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb55516-be81-4763-af62-e371705317c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz\" (UID: \"8cb55516-be81-4763-af62-e371705317c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:03.260107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.260073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb55516-be81-4763-af62-e371705317c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz\" (UID: \"8cb55516-be81-4763-af62-e371705317c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:03.260670 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.260652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb55516-be81-4763-af62-e371705317c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz\" (UID: \"8cb55516-be81-4763-af62-e371705317c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:03.358871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.358834    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:03.498828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:03.498803    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"]
Jul 18 01:14:03.501256 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:14:03.501218    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb55516_be81_4763_af62_e371705317c2.slice/crio-26a7151b75a900f4594c23bfadf9184664415fbc774b0773dd3deb745c085a46 WatchSource:0}: Error finding container 26a7151b75a900f4594c23bfadf9184664415fbc774b0773dd3deb745c085a46: Status 404 returned error can't find the container with id 26a7151b75a900f4594c23bfadf9184664415fbc774b0773dd3deb745c085a46
Jul 18 01:14:04.124688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:04.124650    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerStarted","Data":"cbd6c9810451415880a11398035b9744489401a96de477b1768ad85b7fc6cac3"}
Jul 18 01:14:04.125104 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:04.124704    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerStarted","Data":"26a7151b75a900f4594c23bfadf9184664415fbc774b0773dd3deb745c085a46"}
Jul 18 01:14:04.140629 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:04.140574    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" podStartSLOduration=1.140558785 podStartE2EDuration="1.140558785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:14:04.138339891 +0000 UTC m=+2023.665844850" watchObservedRunningTime="2026-07-18 01:14:04.140558785 +0000 UTC m=+2023.668063744"
Jul 18 01:14:05.131009 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:05.130975    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cb55516-be81-4763-af62-e371705317c2" containerID="cbd6c9810451415880a11398035b9744489401a96de477b1768ad85b7fc6cac3" exitCode=6
Jul 18 01:14:05.131548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:05.131067    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerDied","Data":"cbd6c9810451415880a11398035b9744489401a96de477b1768ad85b7fc6cac3"}
Jul 18 01:14:05.131548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:05.131323    2567 scope.go:117] "RemoveContainer" containerID="cbd6c9810451415880a11398035b9744489401a96de477b1768ad85b7fc6cac3"
Jul 18 01:14:06.137859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:06.137819    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cb55516-be81-4763-af62-e371705317c2" containerID="e89a9866f7c0b6950ff13c53d04bd51472510157ceb6db66b4e68d75a5668f9d" exitCode=6
Jul 18 01:14:06.138318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:06.137909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerDied","Data":"e89a9866f7c0b6950ff13c53d04bd51472510157ceb6db66b4e68d75a5668f9d"}
Jul 18 01:14:06.138318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:06.137971    2567 scope.go:117] "RemoveContainer" containerID="cbd6c9810451415880a11398035b9744489401a96de477b1768ad85b7fc6cac3"
Jul 18 01:14:06.138318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:06.138154    2567 scope.go:117] "RemoveContainer" containerID="e89a9866f7c0b6950ff13c53d04bd51472510157ceb6db66b4e68d75a5668f9d"
Jul 18 01:14:06.138458 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:06.138391    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz_odh-ai-gateway-infra(8cb55516-be81-4763-af62-e371705317c2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" podUID="8cb55516-be81-4763-af62-e371705317c2"
Jul 18 01:14:07.144172 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:07.144140    2567 scope.go:117] "RemoveContainer" containerID="e89a9866f7c0b6950ff13c53d04bd51472510157ceb6db66b4e68d75a5668f9d"
Jul 18 01:14:07.144604 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:07.144356    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz_odh-ai-gateway-infra(8cb55516-be81-4763-af62-e371705317c2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" podUID="8cb55516-be81-4763-af62-e371705317c2"
Jul 18 01:14:08.439456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.439421    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"]
Jul 18 01:14:08.439852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.439788    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:08.439852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.439799    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3283e0-1969-493a-ab00-03c7f4f1b979" containerName="revoke-keys"
Jul 18 01:14:08.439852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.439820    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:08.439852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.439826    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d7e4c5e-967f-421c-ac44-3e553301f92b" containerName="revoke-keys"
Jul 18 01:14:08.444249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.444230    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:08.449847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.449812    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"]
Jul 18 01:14:08.509774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.509726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5068dbfd-23bd-4691-81da-5740221399a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn\" (UID: \"5068dbfd-23bd-4691-81da-5740221399a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:08.610456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.610413    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5068dbfd-23bd-4691-81da-5740221399a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn\" (UID: \"5068dbfd-23bd-4691-81da-5740221399a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:08.611061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.611039    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5068dbfd-23bd-4691-81da-5740221399a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn\" (UID: \"5068dbfd-23bd-4691-81da-5740221399a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:08.755555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.755467    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:08.894737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:08.894627    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"]
Jul 18 01:14:08.897532 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:14:08.897505    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5068dbfd_23bd_4691_81da_5740221399a5.slice/crio-f007abfe0f829760ac5cacadd378ade871f65c518bc9ef994b79f3cf63bddc47 WatchSource:0}: Error finding container f007abfe0f829760ac5cacadd378ade871f65c518bc9ef994b79f3cf63bddc47: Status 404 returned error can't find the container with id f007abfe0f829760ac5cacadd378ade871f65c518bc9ef994b79f3cf63bddc47
Jul 18 01:14:09.155226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:09.155187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerStarted","Data":"4138d51bccbd4a7e891ce5859882c7125b9d17ab2ddcb671661de9e71ed4f9d0"}
Jul 18 01:14:09.155450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:09.155242    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerStarted","Data":"f007abfe0f829760ac5cacadd378ade871f65c518bc9ef994b79f3cf63bddc47"}
Jul 18 01:14:09.169824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:09.169768    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" podStartSLOduration=1.169744479 podStartE2EDuration="1.169744479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:14:09.169552705 +0000 UTC m=+2028.697057665" watchObservedRunningTime="2026-07-18 01:14:09.169744479 +0000 UTC m=+2028.697249438"
Jul 18 01:14:10.160741 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.160709    2567 generic.go:356] "Generic (PLEG): container finished" podID="5068dbfd-23bd-4691-81da-5740221399a5" containerID="4138d51bccbd4a7e891ce5859882c7125b9d17ab2ddcb671661de9e71ed4f9d0" exitCode=7
Jul 18 01:14:10.161246 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.160809    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerDied","Data":"4138d51bccbd4a7e891ce5859882c7125b9d17ab2ddcb671661de9e71ed4f9d0"}
Jul 18 01:14:10.161246 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.161022    2567 scope.go:117] "RemoveContainer" containerID="4138d51bccbd4a7e891ce5859882c7125b9d17ab2ddcb671661de9e71ed4f9d0"
Jul 18 01:14:10.560604 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.560571    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"]
Jul 18 01:14:10.564167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.564148    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:10.570900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.570843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"]
Jul 18 01:14:10.629739 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.629697    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/126c8d86-bc0d-431b-a43f-35ecfa436f23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp\" (UID: \"126c8d86-bc0d-431b-a43f-35ecfa436f23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:10.730687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.730644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/126c8d86-bc0d-431b-a43f-35ecfa436f23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp\" (UID: \"126c8d86-bc0d-431b-a43f-35ecfa436f23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:10.731423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.731398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/126c8d86-bc0d-431b-a43f-35ecfa436f23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp\" (UID: \"126c8d86-bc0d-431b-a43f-35ecfa436f23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:10.876605 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:10.876519    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:11.016395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:11.016368    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"]
Jul 18 01:14:11.018919 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:14:11.018887    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126c8d86_bc0d_431b_a43f_35ecfa436f23.slice/crio-ca370d4cf30d516099aef33596d36dfd537faa62c9e198b07ea4391c709c2869 WatchSource:0}: Error finding container ca370d4cf30d516099aef33596d36dfd537faa62c9e198b07ea4391c709c2869: Status 404 returned error can't find the container with id ca370d4cf30d516099aef33596d36dfd537faa62c9e198b07ea4391c709c2869
Jul 18 01:14:11.169393 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:11.169302    2567 generic.go:356] "Generic (PLEG): container finished" podID="5068dbfd-23bd-4691-81da-5740221399a5" containerID="fec5afd4206f4cd69cd26a80c9de18fa67608860ebd4e3d5e9c43ba95aeaac24" exitCode=7
Jul 18 01:14:11.169807 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:11.169386    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerDied","Data":"fec5afd4206f4cd69cd26a80c9de18fa67608860ebd4e3d5e9c43ba95aeaac24"}
Jul 18 01:14:11.169807 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:11.169448    2567 scope.go:117] "RemoveContainer" containerID="4138d51bccbd4a7e891ce5859882c7125b9d17ab2ddcb671661de9e71ed4f9d0"
Jul 18 01:14:11.169807 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:11.169694    2567 scope.go:117] "RemoveContainer" containerID="fec5afd4206f4cd69cd26a80c9de18fa67608860ebd4e3d5e9c43ba95aeaac24"
Jul 18 01:14:11.169966 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:11.169940    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn_odh-ai-gateway-infra(5068dbfd-23bd-4691-81da-5740221399a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" podUID="5068dbfd-23bd-4691-81da-5740221399a5"
Jul 18 01:14:11.170852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:11.170831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" event={"ID":"126c8d86-bc0d-431b-a43f-35ecfa436f23","Type":"ContainerStarted","Data":"ca370d4cf30d516099aef33596d36dfd537faa62c9e198b07ea4391c709c2869"}
Jul 18 01:14:12.177625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:12.177590    2567 generic.go:356] "Generic (PLEG): container finished" podID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerID="b3709ec450872ca43e8f05f8c1cdd504e9f14a6da867c5fc9fda75e87390fe13" exitCode=6
Jul 18 01:14:12.178072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:12.177674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" event={"ID":"126c8d86-bc0d-431b-a43f-35ecfa436f23","Type":"ContainerDied","Data":"b3709ec450872ca43e8f05f8c1cdd504e9f14a6da867c5fc9fda75e87390fe13"}
Jul 18 01:14:12.178072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:12.177853    2567 scope.go:117] "RemoveContainer" containerID="b3709ec450872ca43e8f05f8c1cdd504e9f14a6da867c5fc9fda75e87390fe13"
Jul 18 01:14:12.179575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:12.179557    2567 scope.go:117] "RemoveContainer" containerID="fec5afd4206f4cd69cd26a80c9de18fa67608860ebd4e3d5e9c43ba95aeaac24"
Jul 18 01:14:12.179740 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:12.179723    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn_odh-ai-gateway-infra(5068dbfd-23bd-4691-81da-5740221399a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" podUID="5068dbfd-23bd-4691-81da-5740221399a5"
Jul 18 01:14:13.185557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:13.185527    2567 generic.go:356] "Generic (PLEG): container finished" podID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerID="a003f74ffd14509d7ddd697b1e21e2b22e1de099c3c46b30bce9244471d18f98" exitCode=6
Jul 18 01:14:13.186061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:13.185618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" event={"ID":"126c8d86-bc0d-431b-a43f-35ecfa436f23","Type":"ContainerDied","Data":"a003f74ffd14509d7ddd697b1e21e2b22e1de099c3c46b30bce9244471d18f98"}
Jul 18 01:14:13.186061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:13.185679    2567 scope.go:117] "RemoveContainer" containerID="b3709ec450872ca43e8f05f8c1cdd504e9f14a6da867c5fc9fda75e87390fe13"
Jul 18 01:14:13.186061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:13.185834    2567 scope.go:117] "RemoveContainer" containerID="a003f74ffd14509d7ddd697b1e21e2b22e1de099c3c46b30bce9244471d18f98"
Jul 18 01:14:13.186061 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:13.186059    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp_odh-ai-gateway-infra(126c8d86-bc0d-431b-a43f-35ecfa436f23)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23"
Jul 18 01:14:14.192537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:14.192513    2567 scope.go:117] "RemoveContainer" containerID="a003f74ffd14509d7ddd697b1e21e2b22e1de099c3c46b30bce9244471d18f98"
Jul 18 01:14:14.192921 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:14.192700    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp_odh-ai-gateway-infra(126c8d86-bc0d-431b-a43f-35ecfa436f23)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23"
Jul 18 01:14:21.913929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:21.913899    2567 scope.go:117] "RemoveContainer" containerID="e89a9866f7c0b6950ff13c53d04bd51472510157ceb6db66b4e68d75a5668f9d"
Jul 18 01:14:22.233967 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:22.233871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerStarted","Data":"6597e5062d806ee2a6f4f6d0a7df6866d82a73aafc380dfe59a37cb1b8a7976d"}
Jul 18 01:14:23.241216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:23.241185    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cb55516-be81-4763-af62-e371705317c2" containerID="6597e5062d806ee2a6f4f6d0a7df6866d82a73aafc380dfe59a37cb1b8a7976d" exitCode=6
Jul 18 01:14:23.241883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:23.241250    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerDied","Data":"6597e5062d806ee2a6f4f6d0a7df6866d82a73aafc380dfe59a37cb1b8a7976d"}
Jul 18 01:14:23.241883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:23.241331    2567 scope.go:117] "RemoveContainer" containerID="e89a9866f7c0b6950ff13c53d04bd51472510157ceb6db66b4e68d75a5668f9d"
Jul 18 01:14:23.241883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:23.241631    2567 scope.go:117] "RemoveContainer" containerID="6597e5062d806ee2a6f4f6d0a7df6866d82a73aafc380dfe59a37cb1b8a7976d"
Jul 18 01:14:23.242083 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:23.241929    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz_odh-ai-gateway-infra(8cb55516-be81-4763-af62-e371705317c2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" podUID="8cb55516-be81-4763-af62-e371705317c2"
Jul 18 01:14:23.256933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:23.256905    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"]
Jul 18 01:14:24.390757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:24.390728    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:24.569419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:24.569391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb55516-be81-4763-af62-e371705317c2-maas-api-service-ca\") pod \"8cb55516-be81-4763-af62-e371705317c2\" (UID: \"8cb55516-be81-4763-af62-e371705317c2\") "
Jul 18 01:14:24.569822 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:24.569801    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cb55516-be81-4763-af62-e371705317c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8cb55516-be81-4763-af62-e371705317c2" (UID: "8cb55516-be81-4763-af62-e371705317c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:14:24.671174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:24.671133    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb55516-be81-4763-af62-e371705317c2-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:14:25.254077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:25.253979    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz" event={"ID":"8cb55516-be81-4763-af62-e371705317c2","Type":"ContainerDied","Data":"26a7151b75a900f4594c23bfadf9184664415fbc774b0773dd3deb745c085a46"}
Jul 18 01:14:25.254077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:25.254006    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"
Jul 18 01:14:25.254425 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:25.254090    2567 scope.go:117] "RemoveContainer" containerID="6597e5062d806ee2a6f4f6d0a7df6866d82a73aafc380dfe59a37cb1b8a7976d"
Jul 18 01:14:25.268783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:25.268743    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"]
Jul 18 01:14:25.275943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:25.275915    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-x2jcz"]
Jul 18 01:14:25.913779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:25.913738    2567 scope.go:117] "RemoveContainer" containerID="a003f74ffd14509d7ddd697b1e21e2b22e1de099c3c46b30bce9244471d18f98"
Jul 18 01:14:26.913263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:26.913178    2567 scope.go:117] "RemoveContainer" containerID="fec5afd4206f4cd69cd26a80c9de18fa67608860ebd4e3d5e9c43ba95aeaac24"
Jul 18 01:14:26.919720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:26.919694    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cb55516-be81-4763-af62-e371705317c2" path="/var/lib/kubelet/pods/8cb55516-be81-4763-af62-e371705317c2/volumes"
Jul 18 01:14:27.268225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:27.268133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerStarted","Data":"b191cee97d09e575dcf5a36fada9973fe3363302c98719838e561bd957bcbcf2"}
Jul 18 01:14:27.269976 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:27.269946    2567 generic.go:356] "Generic (PLEG): container finished" podID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerID="5513be0e6f5659c51088d1cec705007f3dc2ec01334355875c3c72c27254f603" exitCode=6
Jul 18 01:14:27.270124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:27.270014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" event={"ID":"126c8d86-bc0d-431b-a43f-35ecfa436f23","Type":"ContainerDied","Data":"5513be0e6f5659c51088d1cec705007f3dc2ec01334355875c3c72c27254f603"}
Jul 18 01:14:27.270124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:27.270063    2567 scope.go:117] "RemoveContainer" containerID="a003f74ffd14509d7ddd697b1e21e2b22e1de099c3c46b30bce9244471d18f98"
Jul 18 01:14:27.270344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:27.270327    2567 scope.go:117] "RemoveContainer" containerID="5513be0e6f5659c51088d1cec705007f3dc2ec01334355875c3c72c27254f603"
Jul 18 01:14:27.270567 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:27.270546    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp_odh-ai-gateway-infra(126c8d86-bc0d-431b-a43f-35ecfa436f23)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23"
Jul 18 01:14:28.278994 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.278963    2567 generic.go:356] "Generic (PLEG): container finished" podID="5068dbfd-23bd-4691-81da-5740221399a5" containerID="b191cee97d09e575dcf5a36fada9973fe3363302c98719838e561bd957bcbcf2" exitCode=7
Jul 18 01:14:28.279448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.279040    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerDied","Data":"b191cee97d09e575dcf5a36fada9973fe3363302c98719838e561bd957bcbcf2"}
Jul 18 01:14:28.279448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.279101    2567 scope.go:117] "RemoveContainer" containerID="fec5afd4206f4cd69cd26a80c9de18fa67608860ebd4e3d5e9c43ba95aeaac24"
Jul 18 01:14:28.279448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.279437    2567 scope.go:117] "RemoveContainer" containerID="b191cee97d09e575dcf5a36fada9973fe3363302c98719838e561bd957bcbcf2"
Jul 18 01:14:28.279694 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:14:28.279673    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn_odh-ai-gateway-infra(5068dbfd-23bd-4691-81da-5740221399a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" podUID="5068dbfd-23bd-4691-81da-5740221399a5"
Jul 18 01:14:28.293174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.293149    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"]
Jul 18 01:14:28.320402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.320369    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"]
Jul 18 01:14:28.460947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.460922    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:28.608154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.608122    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/126c8d86-bc0d-431b-a43f-35ecfa436f23-maas-api-service-ca\") pod \"126c8d86-bc0d-431b-a43f-35ecfa436f23\" (UID: \"126c8d86-bc0d-431b-a43f-35ecfa436f23\") "
Jul 18 01:14:28.608537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.608515    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126c8d86-bc0d-431b-a43f-35ecfa436f23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "126c8d86-bc0d-431b-a43f-35ecfa436f23" (UID: "126c8d86-bc0d-431b-a43f-35ecfa436f23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:14:28.709127 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:28.709093    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/126c8d86-bc0d-431b-a43f-35ecfa436f23-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:14:29.287183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.287152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp" event={"ID":"126c8d86-bc0d-431b-a43f-35ecfa436f23","Type":"ContainerDied","Data":"ca370d4cf30d516099aef33596d36dfd537faa62c9e198b07ea4391c709c2869"}
Jul 18 01:14:29.287582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.287180    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"
Jul 18 01:14:29.287582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.287219    2567 scope.go:117] "RemoveContainer" containerID="5513be0e6f5659c51088d1cec705007f3dc2ec01334355875c3c72c27254f603"
Jul 18 01:14:29.311693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.311652    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"]
Jul 18 01:14:29.318785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.318337    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-nr5wp"]
Jul 18 01:14:29.430607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.430577    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:29.616331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.616303    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5068dbfd-23bd-4691-81da-5740221399a5-maas-api-service-ca\") pod \"5068dbfd-23bd-4691-81da-5740221399a5\" (UID: \"5068dbfd-23bd-4691-81da-5740221399a5\") "
Jul 18 01:14:29.616723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.616701    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5068dbfd-23bd-4691-81da-5740221399a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5068dbfd-23bd-4691-81da-5740221399a5" (UID: "5068dbfd-23bd-4691-81da-5740221399a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:14:29.717407 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:29.717373    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5068dbfd-23bd-4691-81da-5740221399a5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:14:30.293472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.293440    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"
Jul 18 01:14:30.294033 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.293476    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn" event={"ID":"5068dbfd-23bd-4691-81da-5740221399a5","Type":"ContainerDied","Data":"f007abfe0f829760ac5cacadd378ade871f65c518bc9ef994b79f3cf63bddc47"}
Jul 18 01:14:30.294033 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.293547    2567 scope.go:117] "RemoveContainer" containerID="b191cee97d09e575dcf5a36fada9973fe3363302c98719838e561bd957bcbcf2"
Jul 18 01:14:30.315510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.315479    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"]
Jul 18 01:14:30.325600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.325571    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wrlhn"]
Jul 18 01:14:30.918680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.918644    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" path="/var/lib/kubelet/pods/126c8d86-bc0d-431b-a43f-35ecfa436f23/volumes"
Jul 18 01:14:30.919090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:14:30.919068    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5068dbfd-23bd-4691-81da-5740221399a5" path="/var/lib/kubelet/pods/5068dbfd-23bd-4691-81da-5740221399a5/volumes"
Jul 18 01:15:00.141968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.141928    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"]
Jul 18 01:15:00.142566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142497    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.142566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142518    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.142566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142532    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.142566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142540    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.142566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142559    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142568    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142577    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142585    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142613    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142621    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142634    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142644    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142657    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142668    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142758    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142770    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142781    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142791    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142802    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142811    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142820    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.142884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.142829    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.146032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146009    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"]
Jul 18 01:15:00.146167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"
Jul 18 01:15:00.146452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146426    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.146452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146449    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="126c8d86-bc0d-431b-a43f-35ecfa436f23" containerName="revoke-keys"
Jul 18 01:15:00.146640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146467    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.146640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146473    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5068dbfd-23bd-4691-81da-5740221399a5" containerName="revoke-keys"
Jul 18 01:15:00.146640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.146553    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb55516-be81-4763-af62-e371705317c2" containerName="revoke-keys"
Jul 18 01:15:00.148740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.148723    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:15:00.149341 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.149327    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"
Jul 18 01:15:00.153919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.153895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"]
Jul 18 01:15:00.158688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.158666    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"]
Jul 18 01:15:00.158874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.158724    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"
Jul 18 01:15:00.167177 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.167155    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"
Jul 18 01:15:00.315579 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.315553    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"]
Jul 18 01:15:00.449371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.449287    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" event={"ID":"b8153bd4-2b57-43ca-a416-59f167d71e36","Type":"ContainerStarted","Data":"ed61b0e9196b5c0fc9772f4a13623e0a5c2e5aefc399f080e7e80a78f5e4e55a"}
Jul 18 01:15:00.549077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.549052    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"]
Jul 18 01:15:00.551602 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:15:00.551571    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02928864_fdb6_41ae_af56_6ee7cb81dc39.slice/crio-6de6e7855964eefb8a96fc7e741de7c736f080afcae332acd7b8c8669f0771cf WatchSource:0}: Error finding container 6de6e7855964eefb8a96fc7e741de7c736f080afcae332acd7b8c8669f0771cf: Status 404 returned error can't find the container with id 6de6e7855964eefb8a96fc7e741de7c736f080afcae332acd7b8c8669f0771cf
Jul 18 01:15:00.605532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.605501    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"]
Jul 18 01:15:00.609169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.609150    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:00.611441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.611411    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:15:00.615504 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.615449    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"]
Jul 18 01:15:00.678916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.678884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7734220a-4bdf-4074-b9b9-c8a2bbea53a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v\" (UID: \"7734220a-4bdf-4074-b9b9-c8a2bbea53a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:00.780326 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.780300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7734220a-4bdf-4074-b9b9-c8a2bbea53a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v\" (UID: \"7734220a-4bdf-4074-b9b9-c8a2bbea53a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:00.780948 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.780924    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7734220a-4bdf-4074-b9b9-c8a2bbea53a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v\" (UID: \"7734220a-4bdf-4074-b9b9-c8a2bbea53a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:00.922220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:00.922187    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:01.060355 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.060329    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"]
Jul 18 01:15:01.061965 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:15:01.061934    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7734220a_4bdf_4074_b9b9_c8a2bbea53a1.slice/crio-b52a200ede7407aed84f5d54cad19ba64bfeb49d1e813fbfbb964d6931e439ad WatchSource:0}: Error finding container b52a200ede7407aed84f5d54cad19ba64bfeb49d1e813fbfbb964d6931e439ad: Status 404 returned error can't find the container with id b52a200ede7407aed84f5d54cad19ba64bfeb49d1e813fbfbb964d6931e439ad
Jul 18 01:15:01.456560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.456527    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerID="ac601b8d6cad4dab94ba6b1e3cc689e3499297b2abc35f2ad6f88d9d44c56cd9" exitCode=7
Jul 18 01:15:01.457062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.456621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" event={"ID":"b8153bd4-2b57-43ca-a416-59f167d71e36","Type":"ContainerDied","Data":"ac601b8d6cad4dab94ba6b1e3cc689e3499297b2abc35f2ad6f88d9d44c56cd9"}
Jul 18 01:15:01.457062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.456771    2567 scope.go:117] "RemoveContainer" containerID="ac601b8d6cad4dab94ba6b1e3cc689e3499297b2abc35f2ad6f88d9d44c56cd9"
Jul 18 01:15:01.458322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.458299    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" event={"ID":"7734220a-4bdf-4074-b9b9-c8a2bbea53a1","Type":"ContainerStarted","Data":"951d4b0f8ebc8b713c1dbac905f08b1460cec886855f72a770621623384666bc"}
Jul 18 01:15:01.458442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.458348    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" event={"ID":"7734220a-4bdf-4074-b9b9-c8a2bbea53a1","Type":"ContainerStarted","Data":"b52a200ede7407aed84f5d54cad19ba64bfeb49d1e813fbfbb964d6931e439ad"}
Jul 18 01:15:01.460114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.460081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf" event={"ID":"02928864-fdb6-41ae-af56-6ee7cb81dc39","Type":"ContainerStarted","Data":"4123528ddf385d79ab90a7928dc42ecf408674bfcb2658b92f1ccd77a621eab7"}
Jul 18 01:15:01.460231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.460183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf" event={"ID":"02928864-fdb6-41ae-af56-6ee7cb81dc39","Type":"ContainerStarted","Data":"6de6e7855964eefb8a96fc7e741de7c736f080afcae332acd7b8c8669f0771cf"}
Jul 18 01:15:01.487706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.487652    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" podStartSLOduration=1.487634861 podStartE2EDuration="1.487634861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:15:01.486352812 +0000 UTC m=+2081.013857775" watchObservedRunningTime="2026-07-18 01:15:01.487634861 +0000 UTC m=+2081.015139799"
Jul 18 01:15:01.499884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:01.499834    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf" podStartSLOduration=1.4998173989999999 podStartE2EDuration="1.499817399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:15:01.499302972 +0000 UTC m=+2081.026807931" watchObservedRunningTime="2026-07-18 01:15:01.499817399 +0000 UTC m=+2081.027322358"
Jul 18 01:15:02.467073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.466972    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerID="41c10adf99f79726e4bbe1db448ae24b283dfd336f98789d9d45fd7b419dc2ec" exitCode=7
Jul 18 01:15:02.467565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.467063    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" event={"ID":"b8153bd4-2b57-43ca-a416-59f167d71e36","Type":"ContainerDied","Data":"41c10adf99f79726e4bbe1db448ae24b283dfd336f98789d9d45fd7b419dc2ec"}
Jul 18 01:15:02.467565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.467139    2567 scope.go:117] "RemoveContainer" containerID="ac601b8d6cad4dab94ba6b1e3cc689e3499297b2abc35f2ad6f88d9d44c56cd9"
Jul 18 01:15:02.467565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.467246    2567 scope.go:117] "RemoveContainer" containerID="41c10adf99f79726e4bbe1db448ae24b283dfd336f98789d9d45fd7b419dc2ec"
Jul 18 01:15:02.467565 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:02.467517    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s_odh-ai-gateway-infra(b8153bd4-2b57-43ca-a416-59f167d71e36)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36"
Jul 18 01:15:02.468716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.468649    2567 generic.go:356] "Generic (PLEG): container finished" podID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerID="951d4b0f8ebc8b713c1dbac905f08b1460cec886855f72a770621623384666bc" exitCode=6
Jul 18 01:15:02.468813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.468728    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" event={"ID":"7734220a-4bdf-4074-b9b9-c8a2bbea53a1","Type":"ContainerDied","Data":"951d4b0f8ebc8b713c1dbac905f08b1460cec886855f72a770621623384666bc"}
Jul 18 01:15:02.468903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.468887    2567 scope.go:117] "RemoveContainer" containerID="951d4b0f8ebc8b713c1dbac905f08b1460cec886855f72a770621623384666bc"
Jul 18 01:15:02.470187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.470147    2567 generic.go:356] "Generic (PLEG): container finished" podID="02928864-fdb6-41ae-af56-6ee7cb81dc39" containerID="4123528ddf385d79ab90a7928dc42ecf408674bfcb2658b92f1ccd77a621eab7" exitCode=0
Jul 18 01:15:02.470262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:02.470207    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf" event={"ID":"02928864-fdb6-41ae-af56-6ee7cb81dc39","Type":"ContainerDied","Data":"4123528ddf385d79ab90a7928dc42ecf408674bfcb2658b92f1ccd77a621eab7"}
Jul 18 01:15:03.080540 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.080505    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"]
Jul 18 01:15:03.084047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.084027    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:03.090852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.090531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"]
Jul 18 01:15:03.201183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.201150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk\" (UID: \"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:03.302441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.302405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk\" (UID: \"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:03.303032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.303011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk\" (UID: \"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:03.395680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.395591    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:03.477629 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.477589    2567 scope.go:117] "RemoveContainer" containerID="41c10adf99f79726e4bbe1db448ae24b283dfd336f98789d9d45fd7b419dc2ec"
Jul 18 01:15:03.478092 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:03.477862    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s_odh-ai-gateway-infra(b8153bd4-2b57-43ca-a416-59f167d71e36)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36"
Jul 18 01:15:03.479362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.479332    2567 generic.go:356] "Generic (PLEG): container finished" podID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerID="8affb41a2360ea8ff1dd18e38e40cde15724a780cecd90eb4a89ec983249a8af" exitCode=6
Jul 18 01:15:03.479476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.479401    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" event={"ID":"7734220a-4bdf-4074-b9b9-c8a2bbea53a1","Type":"ContainerDied","Data":"8affb41a2360ea8ff1dd18e38e40cde15724a780cecd90eb4a89ec983249a8af"}
Jul 18 01:15:03.479476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.479467    2567 scope.go:117] "RemoveContainer" containerID="951d4b0f8ebc8b713c1dbac905f08b1460cec886855f72a770621623384666bc"
Jul 18 01:15:03.479692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.479664    2567 scope.go:117] "RemoveContainer" containerID="8affb41a2360ea8ff1dd18e38e40cde15724a780cecd90eb4a89ec983249a8af"
Jul 18 01:15:03.480233 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:03.479892    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v_odh-ai-gateway-infra(7734220a-4bdf-4074-b9b9-c8a2bbea53a1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1"
Jul 18 01:15:03.541500 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.541457    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"]
Jul 18 01:15:03.543463 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:15:03.543422    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1a1f9f_f88e_49c3_8cc5_24da7ab0f218.slice/crio-aed0b83f203da937a0232b30531e9a4a5c29e9e8518f119a9b42590079fba127 WatchSource:0}: Error finding container aed0b83f203da937a0232b30531e9a4a5c29e9e8518f119a9b42590079fba127: Status 404 returned error can't find the container with id aed0b83f203da937a0232b30531e9a4a5c29e9e8518f119a9b42590079fba127
Jul 18 01:15:03.626098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:03.626069    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"
Jul 18 01:15:04.486710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.486683    2567 scope.go:117] "RemoveContainer" containerID="8affb41a2360ea8ff1dd18e38e40cde15724a780cecd90eb4a89ec983249a8af"
Jul 18 01:15:04.487130 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:04.486910    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v_odh-ai-gateway-infra(7734220a-4bdf-4074-b9b9-c8a2bbea53a1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1"
Jul 18 01:15:04.487853 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.487835    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf"
Jul 18 01:15:04.487954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.487838    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738955-jxgcf" event={"ID":"02928864-fdb6-41ae-af56-6ee7cb81dc39","Type":"ContainerDied","Data":"6de6e7855964eefb8a96fc7e741de7c736f080afcae332acd7b8c8669f0771cf"}
Jul 18 01:15:04.488016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.487958    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6de6e7855964eefb8a96fc7e741de7c736f080afcae332acd7b8c8669f0771cf"
Jul 18 01:15:04.489333 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.489311    2567 generic.go:356] "Generic (PLEG): container finished" podID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerID="6b87fe86f5db3f02fb390aea9d75307b97bb2c524b576660c449223687509466" exitCode=6
Jul 18 01:15:04.489457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.489360    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" event={"ID":"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218","Type":"ContainerDied","Data":"6b87fe86f5db3f02fb390aea9d75307b97bb2c524b576660c449223687509466"}
Jul 18 01:15:04.489457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.489397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" event={"ID":"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218","Type":"ContainerStarted","Data":"aed0b83f203da937a0232b30531e9a4a5c29e9e8518f119a9b42590079fba127"}
Jul 18 01:15:04.489595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:04.489579    2567 scope.go:117] "RemoveContainer" containerID="6b87fe86f5db3f02fb390aea9d75307b97bb2c524b576660c449223687509466"
Jul 18 01:15:05.497030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:05.496999    2567 generic.go:356] "Generic (PLEG): container finished" podID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerID="78b3587f6a897d55f5fe05769aa777a32bf8c48f3557933d8c75729a87d0780e" exitCode=6
Jul 18 01:15:05.497030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:05.497035    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" event={"ID":"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218","Type":"ContainerDied","Data":"78b3587f6a897d55f5fe05769aa777a32bf8c48f3557933d8c75729a87d0780e"}
Jul 18 01:15:05.497593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:05.497080    2567 scope.go:117] "RemoveContainer" containerID="6b87fe86f5db3f02fb390aea9d75307b97bb2c524b576660c449223687509466"
Jul 18 01:15:05.497593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:05.497410    2567 scope.go:117] "RemoveContainer" containerID="78b3587f6a897d55f5fe05769aa777a32bf8c48f3557933d8c75729a87d0780e"
Jul 18 01:15:05.497721 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:05.497696    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk_odh-ai-gateway-infra(cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218"
Jul 18 01:15:06.504710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:06.504685    2567 scope.go:117] "RemoveContainer" containerID="78b3587f6a897d55f5fe05769aa777a32bf8c48f3557933d8c75729a87d0780e"
Jul 18 01:15:06.505087 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:06.504885    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk_odh-ai-gateway-infra(cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218"
Jul 18 01:15:08.486432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.486391    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"]
Jul 18 01:15:08.486943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.486919    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02928864-fdb6-41ae-af56-6ee7cb81dc39" containerName="cleanup"
Jul 18 01:15:08.486943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.486943    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="02928864-fdb6-41ae-af56-6ee7cb81dc39" containerName="cleanup"
Jul 18 01:15:08.487046 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.487031    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="02928864-fdb6-41ae-af56-6ee7cb81dc39" containerName="cleanup"
Jul 18 01:15:08.491400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.491384    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:08.496229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.496198    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"]
Jul 18 01:15:08.649974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.649938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bdc136b-62ed-4bab-8a82-488c5729d74a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq\" (UID: \"5bdc136b-62ed-4bab-8a82-488c5729d74a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:08.750983 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.750894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bdc136b-62ed-4bab-8a82-488c5729d74a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq\" (UID: \"5bdc136b-62ed-4bab-8a82-488c5729d74a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:08.751572 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.751551    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bdc136b-62ed-4bab-8a82-488c5729d74a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq\" (UID: \"5bdc136b-62ed-4bab-8a82-488c5729d74a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:08.804185 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.804136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:08.941709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:08.941672    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"]
Jul 18 01:15:08.945317 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:15:08.945263    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bdc136b_62ed_4bab_8a82_488c5729d74a.slice/crio-5cad00cefdb0f26db161fcb0c9e341123e8335a23edbe3b60f15903e0f017080 WatchSource:0}: Error finding container 5cad00cefdb0f26db161fcb0c9e341123e8335a23edbe3b60f15903e0f017080: Status 404 returned error can't find the container with id 5cad00cefdb0f26db161fcb0c9e341123e8335a23edbe3b60f15903e0f017080
Jul 18 01:15:09.521206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:09.521114    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" event={"ID":"5bdc136b-62ed-4bab-8a82-488c5729d74a","Type":"ContainerStarted","Data":"58e7ed16708d757dfa36759be9c85e31fe54f060f52f9f0f9038fab62ed5445c"}
Jul 18 01:15:09.521206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:09.521178    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" event={"ID":"5bdc136b-62ed-4bab-8a82-488c5729d74a","Type":"ContainerStarted","Data":"5cad00cefdb0f26db161fcb0c9e341123e8335a23edbe3b60f15903e0f017080"}
Jul 18 01:15:09.536083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:09.536037    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" podStartSLOduration=1.53602204 podStartE2EDuration="1.53602204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:15:09.53509821 +0000 UTC m=+2089.062603171" watchObservedRunningTime="2026-07-18 01:15:09.53602204 +0000 UTC m=+2089.063527052"
Jul 18 01:15:10.528089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:10.528054    2567 generic.go:356] "Generic (PLEG): container finished" podID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerID="58e7ed16708d757dfa36759be9c85e31fe54f060f52f9f0f9038fab62ed5445c" exitCode=7
Jul 18 01:15:10.528621 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:10.528141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" event={"ID":"5bdc136b-62ed-4bab-8a82-488c5729d74a","Type":"ContainerDied","Data":"58e7ed16708d757dfa36759be9c85e31fe54f060f52f9f0f9038fab62ed5445c"}
Jul 18 01:15:10.528621 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:10.528408    2567 scope.go:117] "RemoveContainer" containerID="58e7ed16708d757dfa36759be9c85e31fe54f060f52f9f0f9038fab62ed5445c"
Jul 18 01:15:11.536187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:11.536094    2567 generic.go:356] "Generic (PLEG): container finished" podID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerID="4d78a43c50e2b57694373c6b50964f3345821031b24eeace88673fbd54b11154" exitCode=7
Jul 18 01:15:11.536187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:11.536159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" event={"ID":"5bdc136b-62ed-4bab-8a82-488c5729d74a","Type":"ContainerDied","Data":"4d78a43c50e2b57694373c6b50964f3345821031b24eeace88673fbd54b11154"}
Jul 18 01:15:11.536687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:11.536209    2567 scope.go:117] "RemoveContainer" containerID="58e7ed16708d757dfa36759be9c85e31fe54f060f52f9f0f9038fab62ed5445c"
Jul 18 01:15:11.536687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:11.536411    2567 scope.go:117] "RemoveContainer" containerID="4d78a43c50e2b57694373c6b50964f3345821031b24eeace88673fbd54b11154"
Jul 18 01:15:11.536687 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:11.536632    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq_odh-ai-gateway-infra(5bdc136b-62ed-4bab-8a82-488c5729d74a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a"
Jul 18 01:15:12.543548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:12.543522    2567 scope.go:117] "RemoveContainer" containerID="4d78a43c50e2b57694373c6b50964f3345821031b24eeace88673fbd54b11154"
Jul 18 01:15:12.543996 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:12.543797    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq_odh-ai-gateway-infra(5bdc136b-62ed-4bab-8a82-488c5729d74a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a"
Jul 18 01:15:15.913497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:15.913470    2567 scope.go:117] "RemoveContainer" containerID="41c10adf99f79726e4bbe1db448ae24b283dfd336f98789d9d45fd7b419dc2ec"
Jul 18 01:15:16.566415 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:16.566389    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerID="ba8bb6c279c33cc74a486aa5e09456294d35b8f0dbc0d96089aa64b241a50248" exitCode=7
Jul 18 01:15:16.566530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:16.566454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" event={"ID":"b8153bd4-2b57-43ca-a416-59f167d71e36","Type":"ContainerDied","Data":"ba8bb6c279c33cc74a486aa5e09456294d35b8f0dbc0d96089aa64b241a50248"}
Jul 18 01:15:16.566570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:16.566543    2567 scope.go:117] "RemoveContainer" containerID="41c10adf99f79726e4bbe1db448ae24b283dfd336f98789d9d45fd7b419dc2ec"
Jul 18 01:15:16.566822 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:16.566806    2567 scope.go:117] "RemoveContainer" containerID="ba8bb6c279c33cc74a486aa5e09456294d35b8f0dbc0d96089aa64b241a50248"
Jul 18 01:15:16.567083 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:16.567064    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s_odh-ai-gateway-infra(b8153bd4-2b57-43ca-a416-59f167d71e36)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36"
Jul 18 01:15:16.913346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:16.913311    2567 scope.go:117] "RemoveContainer" containerID="8affb41a2360ea8ff1dd18e38e40cde15724a780cecd90eb4a89ec983249a8af"
Jul 18 01:15:17.575306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.575252    2567 generic.go:356] "Generic (PLEG): container finished" podID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerID="47e335d242a231cc01dff6f8044cd8517bd3b39840e9a88cf15be61019cb060f" exitCode=6
Jul 18 01:15:17.575306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.575290    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" event={"ID":"7734220a-4bdf-4074-b9b9-c8a2bbea53a1","Type":"ContainerDied","Data":"47e335d242a231cc01dff6f8044cd8517bd3b39840e9a88cf15be61019cb060f"}
Jul 18 01:15:17.575751 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.575362    2567 scope.go:117] "RemoveContainer" containerID="8affb41a2360ea8ff1dd18e38e40cde15724a780cecd90eb4a89ec983249a8af"
Jul 18 01:15:17.575751 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.575675    2567 scope.go:117] "RemoveContainer" containerID="47e335d242a231cc01dff6f8044cd8517bd3b39840e9a88cf15be61019cb060f"
Jul 18 01:15:17.575923 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:17.575899    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v_odh-ai-gateway-infra(7734220a-4bdf-4074-b9b9-c8a2bbea53a1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1"
Jul 18 01:15:17.588977 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.588950    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"]
Jul 18 01:15:17.731995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.731969    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"
Jul 18 01:15:17.913375 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:17.913333    2567 scope.go:117] "RemoveContainer" containerID="78b3587f6a897d55f5fe05769aa777a32bf8c48f3557933d8c75729a87d0780e"
Jul 18 01:15:18.582773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.582740    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" event={"ID":"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218","Type":"ContainerStarted","Data":"d96765b822522c7def482f4a05d216038eb009d64cbd246ff4008130020272ff"}
Jul 18 01:15:18.583994 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.583976    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"
Jul 18 01:15:18.583994 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.583982    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s" event={"ID":"b8153bd4-2b57-43ca-a416-59f167d71e36","Type":"ContainerDied","Data":"ed61b0e9196b5c0fc9772f4a13623e0a5c2e5aefc399f080e7e80a78f5e4e55a"}
Jul 18 01:15:18.584191 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.584034    2567 scope.go:117] "RemoveContainer" containerID="ba8bb6c279c33cc74a486aa5e09456294d35b8f0dbc0d96089aa64b241a50248"
Jul 18 01:15:18.599292 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.599224    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" podStartSLOduration=15.599210474 podStartE2EDuration="15.599210474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:15:18.597738886 +0000 UTC m=+2098.125243847" watchObservedRunningTime="2026-07-18 01:15:18.599210474 +0000 UTC m=+2098.126715435"
Jul 18 01:15:18.602538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.602515    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"]
Jul 18 01:15:18.615079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.615049    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"]
Jul 18 01:15:18.623398 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.623365    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5466ad4-29738955-ncj7s"]
Jul 18 01:15:18.744395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.744370    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:18.836551 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.836452    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7734220a-4bdf-4074-b9b9-c8a2bbea53a1-maas-api-service-ca\") pod \"7734220a-4bdf-4074-b9b9-c8a2bbea53a1\" (UID: \"7734220a-4bdf-4074-b9b9-c8a2bbea53a1\") "
Jul 18 01:15:18.836804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.836781    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7734220a-4bdf-4074-b9b9-c8a2bbea53a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7734220a-4bdf-4074-b9b9-c8a2bbea53a1" (UID: "7734220a-4bdf-4074-b9b9-c8a2bbea53a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:15:18.919347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.919316    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" path="/var/lib/kubelet/pods/b8153bd4-2b57-43ca-a416-59f167d71e36/volumes"
Jul 18 01:15:18.937437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:18.937411    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7734220a-4bdf-4074-b9b9-c8a2bbea53a1-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:15:19.593363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.593330    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"
Jul 18 01:15:19.593363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.593341    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v" event={"ID":"7734220a-4bdf-4074-b9b9-c8a2bbea53a1","Type":"ContainerDied","Data":"b52a200ede7407aed84f5d54cad19ba64bfeb49d1e813fbfbb964d6931e439ad"}
Jul 18 01:15:19.593950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.593442    2567 scope.go:117] "RemoveContainer" containerID="47e335d242a231cc01dff6f8044cd8517bd3b39840e9a88cf15be61019cb060f"
Jul 18 01:15:19.595233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.595207    2567 generic.go:356] "Generic (PLEG): container finished" podID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerID="d96765b822522c7def482f4a05d216038eb009d64cbd246ff4008130020272ff" exitCode=6
Jul 18 01:15:19.595395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.595256    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" event={"ID":"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218","Type":"ContainerDied","Data":"d96765b822522c7def482f4a05d216038eb009d64cbd246ff4008130020272ff"}
Jul 18 01:15:19.595729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.595631    2567 scope.go:117] "RemoveContainer" containerID="d96765b822522c7def482f4a05d216038eb009d64cbd246ff4008130020272ff"
Jul 18 01:15:19.595948 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:19.595922    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk_odh-ai-gateway-infra(cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218"
Jul 18 01:15:19.602479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.602463    2567 scope.go:117] "RemoveContainer" containerID="78b3587f6a897d55f5fe05769aa777a32bf8c48f3557933d8c75729a87d0780e"
Jul 18 01:15:19.607598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.607575    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"]
Jul 18 01:15:19.631802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.631769    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"]
Jul 18 01:15:19.637362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:19.637340    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wcq6v"]
Jul 18 01:15:20.759479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:20.759455    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:20.857192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:20.857119    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218-maas-api-service-ca\") pod \"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218\" (UID: \"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218\") "
Jul 18 01:15:20.857525 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:20.857504    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" (UID: "cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:15:20.919947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:20.919915    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" path="/var/lib/kubelet/pods/7734220a-4bdf-4074-b9b9-c8a2bbea53a1/volumes"
Jul 18 01:15:20.958204 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:20.958167    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:15:21.007028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.007001    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:15:21.015178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.015154    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:15:21.612034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.611996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk" event={"ID":"cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218","Type":"ContainerDied","Data":"aed0b83f203da937a0232b30531e9a4a5c29e9e8518f119a9b42590079fba127"}
Jul 18 01:15:21.612226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.612039    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"
Jul 18 01:15:21.612226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.612089    2567 scope.go:117] "RemoveContainer" containerID="d96765b822522c7def482f4a05d216038eb009d64cbd246ff4008130020272ff"
Jul 18 01:15:21.627176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.627137    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"]
Jul 18 01:15:21.634034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:21.634005    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-f2fwk"]
Jul 18 01:15:22.919833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:22.919801    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" path="/var/lib/kubelet/pods/cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218/volumes"
Jul 18 01:15:26.913732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:26.913690    2567 scope.go:117] "RemoveContainer" containerID="4d78a43c50e2b57694373c6b50964f3345821031b24eeace88673fbd54b11154"
Jul 18 01:15:27.645593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:27.645562    2567 generic.go:356] "Generic (PLEG): container finished" podID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerID="63be72a9f544b8ecade5bb58a12218a9d19357438cb7bbb8e5a5175767bd447a" exitCode=7
Jul 18 01:15:27.645772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:27.645644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" event={"ID":"5bdc136b-62ed-4bab-8a82-488c5729d74a","Type":"ContainerDied","Data":"63be72a9f544b8ecade5bb58a12218a9d19357438cb7bbb8e5a5175767bd447a"}
Jul 18 01:15:27.645772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:27.645710    2567 scope.go:117] "RemoveContainer" containerID="4d78a43c50e2b57694373c6b50964f3345821031b24eeace88673fbd54b11154"
Jul 18 01:15:27.646072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:27.646053    2567 scope.go:117] "RemoveContainer" containerID="63be72a9f544b8ecade5bb58a12218a9d19357438cb7bbb8e5a5175767bd447a"
Jul 18 01:15:27.646351 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:27.646324    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq_odh-ai-gateway-infra(5bdc136b-62ed-4bab-8a82-488c5729d74a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a"
Jul 18 01:15:28.669376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:28.669348    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"]
Jul 18 01:15:28.813934 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:28.813906    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:28.831813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:28.831783    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bdc136b-62ed-4bab-8a82-488c5729d74a-maas-api-service-ca\") pod \"5bdc136b-62ed-4bab-8a82-488c5729d74a\" (UID: \"5bdc136b-62ed-4bab-8a82-488c5729d74a\") "
Jul 18 01:15:28.832262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:28.832236    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bdc136b-62ed-4bab-8a82-488c5729d74a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bdc136b-62ed-4bab-8a82-488c5729d74a" (UID: "5bdc136b-62ed-4bab-8a82-488c5729d74a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:15:28.932580 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:28.932559    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bdc136b-62ed-4bab-8a82-488c5729d74a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:15:29.658716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:29.658686    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"
Jul 18 01:15:29.658880 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:29.658724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq" event={"ID":"5bdc136b-62ed-4bab-8a82-488c5729d74a","Type":"ContainerDied","Data":"5cad00cefdb0f26db161fcb0c9e341123e8335a23edbe3b60f15903e0f017080"}
Jul 18 01:15:29.658880 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:29.658789    2567 scope.go:117] "RemoveContainer" containerID="63be72a9f544b8ecade5bb58a12218a9d19357438cb7bbb8e5a5175767bd447a"
Jul 18 01:15:29.673772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:29.673736    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"]
Jul 18 01:15:29.680015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:29.679991    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-f6lnq"]
Jul 18 01:15:30.920205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:30.920173    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" path="/var/lib/kubelet/pods/5bdc136b-62ed-4bab-8a82-488c5729d74a/volumes"
Jul 18 01:15:50.646030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.645995    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"]
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646378    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646390    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646405    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646411    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646418    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646424    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646431    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646436    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646442    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646446    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646455    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646460    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646468    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646473    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646480    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646484    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646493    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.646530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646498    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646568    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646575    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646583    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646588    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646596    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646601    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646605    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646611    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646615    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646621    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.647120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.646627    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bdc136b-62ed-4bab-8a82-488c5729d74a" containerName="revoke-keys"
Jul 18 01:15:50.651045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.651026    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:15:50.653567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.653543    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:15:50.653743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.653718    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:15:50.656989 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.656965    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"]
Jul 18 01:15:50.711359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.711325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6\" (UID: \"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:15:50.812080 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.812044    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6\" (UID: \"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:15:50.812710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.812687    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6\" (UID: \"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:15:50.963097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:50.963008    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:15:51.104197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:51.104169    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"]
Jul 18 01:15:51.106398 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:15:51.106364    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991cf636_9cbb_40fa_a5cf_f8ef6f500ddb.slice/crio-adc3bbfe26c300d89e300abcc2cc93db6efed41864a8b69b5e31082d224b73ad WatchSource:0}: Error finding container adc3bbfe26c300d89e300abcc2cc93db6efed41864a8b69b5e31082d224b73ad: Status 404 returned error can't find the container with id adc3bbfe26c300d89e300abcc2cc93db6efed41864a8b69b5e31082d224b73ad
Jul 18 01:15:51.773187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:51.773139    2567 generic.go:356] "Generic (PLEG): container finished" podID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerID="a4a716aa953a6f05b1162051034da9f80f3aafef12e8339c1eb64c5f044dfc3c" exitCode=6
Jul 18 01:15:51.773590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:51.773256    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" event={"ID":"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb","Type":"ContainerDied","Data":"a4a716aa953a6f05b1162051034da9f80f3aafef12e8339c1eb64c5f044dfc3c"}
Jul 18 01:15:51.773590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:51.773394    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" event={"ID":"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb","Type":"ContainerStarted","Data":"adc3bbfe26c300d89e300abcc2cc93db6efed41864a8b69b5e31082d224b73ad"}
Jul 18 01:15:51.773590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:51.773408    2567 scope.go:117] "RemoveContainer" containerID="a4a716aa953a6f05b1162051034da9f80f3aafef12e8339c1eb64c5f044dfc3c"
Jul 18 01:15:52.780031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:52.780002    2567 generic.go:356] "Generic (PLEG): container finished" podID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerID="efd745c03e7f9d72be82124682392baa5eb12881f7c85b3218af521af4b6a5fa" exitCode=6
Jul 18 01:15:52.780390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:52.780071    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" event={"ID":"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb","Type":"ContainerDied","Data":"efd745c03e7f9d72be82124682392baa5eb12881f7c85b3218af521af4b6a5fa"}
Jul 18 01:15:52.780390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:52.780147    2567 scope.go:117] "RemoveContainer" containerID="a4a716aa953a6f05b1162051034da9f80f3aafef12e8339c1eb64c5f044dfc3c"
Jul 18 01:15:52.780390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:52.780329    2567 scope.go:117] "RemoveContainer" containerID="efd745c03e7f9d72be82124682392baa5eb12881f7c85b3218af521af4b6a5fa"
Jul 18 01:15:52.780603 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:52.780583    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6_odh-ai-gateway-infra(991cf636-9cbb-40fa-a5cf-f8ef6f500ddb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb"
Jul 18 01:15:53.129898 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.129866    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"]
Jul 18 01:15:53.130302 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130288    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:53.130344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130304    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7734220a-4bdf-4074-b9b9-c8a2bbea53a1" containerName="revoke-keys"
Jul 18 01:15:53.130344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130324    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:53.130344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130329    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:53.130344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130339    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:53.130344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130344    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8153bd4-2b57-43ca-a416-59f167d71e36" containerName="cleanup"
Jul 18 01:15:53.130504 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.130397    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1a1f9f-f88e-49c3-8cc5-24da7ab0f218" containerName="revoke-keys"
Jul 18 01:15:53.134583 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.134564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:15:53.143572 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.143544    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"]
Jul 18 01:15:53.234599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.234560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh\" (UID: \"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:15:53.335513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.335473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh\" (UID: \"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:15:53.336120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.336099    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh\" (UID: \"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:15:53.446254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.446137    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:15:53.585702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.585675    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"]
Jul 18 01:15:53.587638 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:15:53.587606    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9908aa_7ec5_4e73_b4c8_6feb0d7a5748.slice/crio-9f7eac75ef686b41620eb71d19445e153c3115dee485ee4ce408c04c59425db4 WatchSource:0}: Error finding container 9f7eac75ef686b41620eb71d19445e153c3115dee485ee4ce408c04c59425db4: Status 404 returned error can't find the container with id 9f7eac75ef686b41620eb71d19445e153c3115dee485ee4ce408c04c59425db4
Jul 18 01:15:53.787111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.787077    2567 scope.go:117] "RemoveContainer" containerID="efd745c03e7f9d72be82124682392baa5eb12881f7c85b3218af521af4b6a5fa"
Jul 18 01:15:53.787546 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:53.787320    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6_odh-ai-gateway-infra(991cf636-9cbb-40fa-a5cf-f8ef6f500ddb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb"
Jul 18 01:15:53.788073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:53.788042    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" event={"ID":"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748","Type":"ContainerStarted","Data":"9f7eac75ef686b41620eb71d19445e153c3115dee485ee4ce408c04c59425db4"}
Jul 18 01:15:54.794995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:54.794959    2567 generic.go:356] "Generic (PLEG): container finished" podID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerID="249c41d843367fe03a76bc80a6758fff887170758a3f4dfbffaed8801c39f0bd" exitCode=6
Jul 18 01:15:54.795450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:54.795053    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" event={"ID":"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748","Type":"ContainerDied","Data":"249c41d843367fe03a76bc80a6758fff887170758a3f4dfbffaed8801c39f0bd"}
Jul 18 01:15:54.795450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:54.795179    2567 scope.go:117] "RemoveContainer" containerID="249c41d843367fe03a76bc80a6758fff887170758a3f4dfbffaed8801c39f0bd"
Jul 18 01:15:55.801954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:55.801858    2567 generic.go:356] "Generic (PLEG): container finished" podID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerID="5640e03ed2bcaa85772090e6ed1057ce564e7904c46f48d6baa94d766ba19e49" exitCode=6
Jul 18 01:15:55.801954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:55.801941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" event={"ID":"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748","Type":"ContainerDied","Data":"5640e03ed2bcaa85772090e6ed1057ce564e7904c46f48d6baa94d766ba19e49"}
Jul 18 01:15:55.802452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:55.802002    2567 scope.go:117] "RemoveContainer" containerID="249c41d843367fe03a76bc80a6758fff887170758a3f4dfbffaed8801c39f0bd"
Jul 18 01:15:55.802452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:55.802189    2567 scope.go:117] "RemoveContainer" containerID="5640e03ed2bcaa85772090e6ed1057ce564e7904c46f48d6baa94d766ba19e49"
Jul 18 01:15:55.802535 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:55.802456    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh_odh-ai-gateway-infra(6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748"
Jul 18 01:15:56.809087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:15:56.809056    2567 scope.go:117] "RemoveContainer" containerID="5640e03ed2bcaa85772090e6ed1057ce564e7904c46f48d6baa94d766ba19e49"
Jul 18 01:15:56.809589 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:15:56.809230    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh_odh-ai-gateway-infra(6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748"
Jul 18 01:16:06.303610 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:06.303563    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:16:06.392186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:06.392153    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:16:07.913808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:07.913775    2567 scope.go:117] "RemoveContainer" containerID="5640e03ed2bcaa85772090e6ed1057ce564e7904c46f48d6baa94d766ba19e49"
Jul 18 01:16:08.526329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.526230    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"]
Jul 18 01:16:08.529726 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.529710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:08.536637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.536604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"]
Jul 18 01:16:08.678408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.678368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p\" (UID: \"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:08.779713 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.779619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p\" (UID: \"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:08.780307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.780252    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p\" (UID: \"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:08.843261 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.843227    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:08.872470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.872441    2567 generic.go:356] "Generic (PLEG): container finished" podID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerID="1ddf7841b131ceb6513ac380c72d8cb6292590c511fa56ba800ad2cdc7d814dd" exitCode=6
Jul 18 01:16:08.872637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.872509    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" event={"ID":"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748","Type":"ContainerDied","Data":"1ddf7841b131ceb6513ac380c72d8cb6292590c511fa56ba800ad2cdc7d814dd"}
Jul 18 01:16:08.872637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.872584    2567 scope.go:117] "RemoveContainer" containerID="5640e03ed2bcaa85772090e6ed1057ce564e7904c46f48d6baa94d766ba19e49"
Jul 18 01:16:08.872863 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.872846    2567 scope.go:117] "RemoveContainer" containerID="1ddf7841b131ceb6513ac380c72d8cb6292590c511fa56ba800ad2cdc7d814dd"
Jul 18 01:16:08.873121 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:08.873098    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh_odh-ai-gateway-infra(6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748"
Jul 18 01:16:08.913850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.913818    2567 scope.go:117] "RemoveContainer" containerID="efd745c03e7f9d72be82124682392baa5eb12881f7c85b3218af521af4b6a5fa"
Jul 18 01:16:08.988779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:08.988655    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"]
Jul 18 01:16:08.991091 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:16:08.991060    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ee0d24_c6d6_4b41_a7c4_a4cb1e96c826.slice/crio-bc0361d67b4e8e4dc6ff3ec0c8e8acea7433c139d557601a7ee660f1f1d2c415 WatchSource:0}: Error finding container bc0361d67b4e8e4dc6ff3ec0c8e8acea7433c139d557601a7ee660f1f1d2c415: Status 404 returned error can't find the container with id bc0361d67b4e8e4dc6ff3ec0c8e8acea7433c139d557601a7ee660f1f1d2c415
Jul 18 01:16:09.018734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.018706    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:16:09.119430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.119394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:16:09.879516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.879432    2567 generic.go:356] "Generic (PLEG): container finished" podID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerID="43e23fd11b02a9fd79a8565bec2e31d9a0a98ef8249873849445c4e108550697" exitCode=7
Jul 18 01:16:09.879688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.879523    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" event={"ID":"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826","Type":"ContainerDied","Data":"43e23fd11b02a9fd79a8565bec2e31d9a0a98ef8249873849445c4e108550697"}
Jul 18 01:16:09.879688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.879585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" event={"ID":"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826","Type":"ContainerStarted","Data":"bc0361d67b4e8e4dc6ff3ec0c8e8acea7433c139d557601a7ee660f1f1d2c415"}
Jul 18 01:16:09.879834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.879711    2567 scope.go:117] "RemoveContainer" containerID="43e23fd11b02a9fd79a8565bec2e31d9a0a98ef8249873849445c4e108550697"
Jul 18 01:16:09.881491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.881468    2567 generic.go:356] "Generic (PLEG): container finished" podID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerID="b5670287e20b56548357820cedc96ca2d91bd098606e2436e547e6212fafb31b" exitCode=6
Jul 18 01:16:09.881601 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.881544    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" event={"ID":"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb","Type":"ContainerDied","Data":"b5670287e20b56548357820cedc96ca2d91bd098606e2436e547e6212fafb31b"}
Jul 18 01:16:09.881601 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.881598    2567 scope.go:117] "RemoveContainer" containerID="efd745c03e7f9d72be82124682392baa5eb12881f7c85b3218af521af4b6a5fa"
Jul 18 01:16:09.881921 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.881896    2567 scope.go:117] "RemoveContainer" containerID="b5670287e20b56548357820cedc96ca2d91bd098606e2436e547e6212fafb31b"
Jul 18 01:16:09.882168 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:09.882134    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6_odh-ai-gateway-infra(991cf636-9cbb-40fa-a5cf-f8ef6f500ddb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb"
Jul 18 01:16:09.899341 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:09.899319    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"]
Jul 18 01:16:10.045521 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.045496    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:16:10.193720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.193688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748-maas-api-service-ca\") pod \"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748\" (UID: \"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748\") "
Jul 18 01:16:10.194152 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.194121    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" (UID: "6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:16:10.295450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.295410    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:16:10.890778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.890746    2567 generic.go:356] "Generic (PLEG): container finished" podID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerID="bd2b15280587d08e66a627f9f10cd744718a9dba45649eb7ca736c2a7f0881ff" exitCode=7
Jul 18 01:16:10.890972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.890842    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" event={"ID":"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826","Type":"ContainerDied","Data":"bd2b15280587d08e66a627f9f10cd744718a9dba45649eb7ca736c2a7f0881ff"}
Jul 18 01:16:10.890972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.890902    2567 scope.go:117] "RemoveContainer" containerID="43e23fd11b02a9fd79a8565bec2e31d9a0a98ef8249873849445c4e108550697"
Jul 18 01:16:10.891188 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.891153    2567 scope.go:117] "RemoveContainer" containerID="bd2b15280587d08e66a627f9f10cd744718a9dba45649eb7ca736c2a7f0881ff"
Jul 18 01:16:10.891440 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:10.891421    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p_odh-ai-gateway-infra(66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826"
Jul 18 01:16:10.896228 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.896208    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"
Jul 18 01:16:10.896358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.896245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh" event={"ID":"6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748","Type":"ContainerDied","Data":"9f7eac75ef686b41620eb71d19445e153c3115dee485ee4ce408c04c59425db4"}
Jul 18 01:16:10.910254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.910236    2567 scope.go:117] "RemoveContainer" containerID="1ddf7841b131ceb6513ac380c72d8cb6292590c511fa56ba800ad2cdc7d814dd"
Jul 18 01:16:10.920632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.920493    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"]
Jul 18 01:16:10.931782 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.931755    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"]
Jul 18 01:16:10.933772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:10.933738    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-lvrwh"]
Jul 18 01:16:11.072192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.072165    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:16:11.209866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.209742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb-maas-api-service-ca\") pod \"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb\" (UID: \"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb\") "
Jul 18 01:16:11.210096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.210075    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" (UID: "991cf636-9cbb-40fa-a5cf-f8ef6f500ddb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:16:11.310522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.310487    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:16:11.903005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.902975    2567 scope.go:117] "RemoveContainer" containerID="bd2b15280587d08e66a627f9f10cd744718a9dba45649eb7ca736c2a7f0881ff"
Jul 18 01:16:11.903227 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:11.903202    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p_odh-ai-gateway-infra(66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826"
Jul 18 01:16:11.904011 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.903991    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"
Jul 18 01:16:11.904128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.904008    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6" event={"ID":"991cf636-9cbb-40fa-a5cf-f8ef6f500ddb","Type":"ContainerDied","Data":"adc3bbfe26c300d89e300abcc2cc93db6efed41864a8b69b5e31082d224b73ad"}
Jul 18 01:16:11.904128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.904064    2567 scope.go:117] "RemoveContainer" containerID="b5670287e20b56548357820cedc96ca2d91bd098606e2436e547e6212fafb31b"
Jul 18 01:16:11.930411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.930370    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"]
Jul 18 01:16:11.938303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:11.938260    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-d66h6"]
Jul 18 01:16:12.919012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:12.918980    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" path="/var/lib/kubelet/pods/6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748/volumes"
Jul 18 01:16:12.919470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:12.919361    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" path="/var/lib/kubelet/pods/991cf636-9cbb-40fa-a5cf-f8ef6f500ddb/volumes"
Jul 18 01:16:25.913704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:25.913676    2567 scope.go:117] "RemoveContainer" containerID="bd2b15280587d08e66a627f9f10cd744718a9dba45649eb7ca736c2a7f0881ff"
Jul 18 01:16:26.984065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:26.984027    2567 generic.go:356] "Generic (PLEG): container finished" podID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerID="66c81c1beae3c435a23aad9c4d9c04909a84fb3051c695fd7c85d3dfaf076ace" exitCode=7
Jul 18 01:16:26.984586 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:26.984109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" event={"ID":"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826","Type":"ContainerDied","Data":"66c81c1beae3c435a23aad9c4d9c04909a84fb3051c695fd7c85d3dfaf076ace"}
Jul 18 01:16:26.984586 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:26.984194    2567 scope.go:117] "RemoveContainer" containerID="bd2b15280587d08e66a627f9f10cd744718a9dba45649eb7ca736c2a7f0881ff"
Jul 18 01:16:26.984586 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:26.984582    2567 scope.go:117] "RemoveContainer" containerID="66c81c1beae3c435a23aad9c4d9c04909a84fb3051c695fd7c85d3dfaf076ace"
Jul 18 01:16:26.984815 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:26.984795    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p_odh-ai-gateway-infra(66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826"
Jul 18 01:16:28.008546 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.008517    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"]
Jul 18 01:16:28.148495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.148474    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:28.159359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.159325    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826-maas-api-service-ca\") pod \"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826\" (UID: \"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826\") "
Jul 18 01:16:28.159638 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.159616    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" (UID: "66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:16:28.260079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.259992    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:16:28.996342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.996310    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"
Jul 18 01:16:28.996342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.996332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p" event={"ID":"66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826","Type":"ContainerDied","Data":"bc0361d67b4e8e4dc6ff3ec0c8e8acea7433c139d557601a7ee660f1f1d2c415"}
Jul 18 01:16:28.996571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:28.996396    2567 scope.go:117] "RemoveContainer" containerID="66c81c1beae3c435a23aad9c4d9c04909a84fb3051c695fd7c85d3dfaf076ace"
Jul 18 01:16:29.011988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:29.011961    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"]
Jul 18 01:16:29.019639 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:29.019614    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dsk7p"]
Jul 18 01:16:30.919682 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:30.919640    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" path="/var/lib/kubelet/pods/66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826/volumes"
Jul 18 01:16:32.687729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:32.687695    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:16:43.171190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171153    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"]
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171545    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171556    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171568    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171573    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171581    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171586    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171593    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:43.171597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171598    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171605    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171610    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171619    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171624    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171633    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171638    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171690    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171696    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171702    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171709    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171716    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="991cf636-9cbb-40fa-a5cf-f8ef6f500ddb" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171720    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:43.171851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.171727    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:43.176012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.175991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:16:43.178581 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.178549    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:16:43.178695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.178639    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:16:43.182580 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.182557    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"]
Jul 18 01:16:43.293427 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.293387    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-xmgls\" (UID: \"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:16:43.393980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.393933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-xmgls\" (UID: \"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:16:43.394663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.394641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-xmgls\" (UID: \"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:16:43.487433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.487339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:16:43.625060 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:43.625033    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"]
Jul 18 01:16:43.627792 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:16:43.627765    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a0c09a_d6e8_42d3_8e27_0ce866d73bb9.slice/crio-185aaa9278448ff0e42837a63a5ebd604481d793114e58820e871f39edbaf2e9 WatchSource:0}: Error finding container 185aaa9278448ff0e42837a63a5ebd604481d793114e58820e871f39edbaf2e9: Status 404 returned error can't find the container with id 185aaa9278448ff0e42837a63a5ebd604481d793114e58820e871f39edbaf2e9
Jul 18 01:16:44.071564 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:44.071527    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" event={"ID":"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9","Type":"ContainerStarted","Data":"6f084a132bc10238d1a8a30796b6cab471f550acc43d57072e7609a21b702dc5"}
Jul 18 01:16:44.071818 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:44.071588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" event={"ID":"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9","Type":"ContainerStarted","Data":"185aaa9278448ff0e42837a63a5ebd604481d793114e58820e871f39edbaf2e9"}
Jul 18 01:16:44.085438 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:44.085386    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" podStartSLOduration=1.085370303 podStartE2EDuration="1.085370303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:16:44.084629019 +0000 UTC m=+2183.612133980" watchObservedRunningTime="2026-07-18 01:16:44.085370303 +0000 UTC m=+2183.612875262"
Jul 18 01:16:45.078181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:45.078145    2567 generic.go:356] "Generic (PLEG): container finished" podID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerID="6f084a132bc10238d1a8a30796b6cab471f550acc43d57072e7609a21b702dc5" exitCode=6
Jul 18 01:16:45.078643 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:45.078215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" event={"ID":"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9","Type":"ContainerDied","Data":"6f084a132bc10238d1a8a30796b6cab471f550acc43d57072e7609a21b702dc5"}
Jul 18 01:16:45.078643 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:45.078471    2567 scope.go:117] "RemoveContainer" containerID="6f084a132bc10238d1a8a30796b6cab471f550acc43d57072e7609a21b702dc5"
Jul 18 01:16:46.084951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:46.084918    2567 generic.go:356] "Generic (PLEG): container finished" podID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerID="5f4518a8085c97ae43fcd75aec8fb941520ce69f99117294134da9707c4e2b71" exitCode=6
Jul 18 01:16:46.085400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:46.084992    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" event={"ID":"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9","Type":"ContainerDied","Data":"5f4518a8085c97ae43fcd75aec8fb941520ce69f99117294134da9707c4e2b71"}
Jul 18 01:16:46.085400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:46.085037    2567 scope.go:117] "RemoveContainer" containerID="6f084a132bc10238d1a8a30796b6cab471f550acc43d57072e7609a21b702dc5"
Jul 18 01:16:46.085400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:46.085348    2567 scope.go:117] "RemoveContainer" containerID="5f4518a8085c97ae43fcd75aec8fb941520ce69f99117294134da9707c4e2b71"
Jul 18 01:16:46.085573 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:46.085554    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-xmgls_odh-ai-gateway-infra(f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9"
Jul 18 01:16:47.091153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:47.091120    2567 scope.go:117] "RemoveContainer" containerID="5f4518a8085c97ae43fcd75aec8fb941520ce69f99117294134da9707c4e2b71"
Jul 18 01:16:47.091648 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:47.091415    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-xmgls_odh-ai-gateway-infra(f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9"
Jul 18 01:16:50.726605 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.726565    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"]
Jul 18 01:16:50.726995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.726939    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:50.726995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.726950    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:50.726995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.726968    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:50.726995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.726974    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:50.727147 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.727039    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ee0d24-c6d6-4b41-a7c4-a4cb1e96c826" containerName="revoke-keys"
Jul 18 01:16:50.727147 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.727052    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9908aa-7ec5-4e73-b4c8-6feb0d7a5748" containerName="revoke-keys"
Jul 18 01:16:50.731310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.731293    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:16:50.739472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.739444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"]
Jul 18 01:16:50.860437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.860403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402d65fb-ab49-44be-b839-a0a8d47cb9bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2\" (UID: \"402d65fb-ab49-44be-b839-a0a8d47cb9bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:16:50.961227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.961188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402d65fb-ab49-44be-b839-a0a8d47cb9bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2\" (UID: \"402d65fb-ab49-44be-b839-a0a8d47cb9bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:16:50.961852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:50.961832    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402d65fb-ab49-44be-b839-a0a8d47cb9bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2\" (UID: \"402d65fb-ab49-44be-b839-a0a8d47cb9bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:16:51.043496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:51.043401    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:16:51.185778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:51.185728    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"]
Jul 18 01:16:51.188181 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:16:51.188156    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod402d65fb_ab49_44be_b839_a0a8d47cb9bc.slice/crio-53a1b23fb879608a6387a481f17fff7fbec94f446c512811121a838245b19ec6 WatchSource:0}: Error finding container 53a1b23fb879608a6387a481f17fff7fbec94f446c512811121a838245b19ec6: Status 404 returned error can't find the container with id 53a1b23fb879608a6387a481f17fff7fbec94f446c512811121a838245b19ec6
Jul 18 01:16:52.119017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:52.118928    2567 generic.go:356] "Generic (PLEG): container finished" podID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerID="d2de8f7724ab8ded5c61401ad9a508169e04bbe623b37251de6471847def7d37" exitCode=6
Jul 18 01:16:52.119017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:52.118991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" event={"ID":"402d65fb-ab49-44be-b839-a0a8d47cb9bc","Type":"ContainerDied","Data":"d2de8f7724ab8ded5c61401ad9a508169e04bbe623b37251de6471847def7d37"}
Jul 18 01:16:52.119460 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:52.119060    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" event={"ID":"402d65fb-ab49-44be-b839-a0a8d47cb9bc","Type":"ContainerStarted","Data":"53a1b23fb879608a6387a481f17fff7fbec94f446c512811121a838245b19ec6"}
Jul 18 01:16:52.119460 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:52.119243    2567 scope.go:117] "RemoveContainer" containerID="d2de8f7724ab8ded5c61401ad9a508169e04bbe623b37251de6471847def7d37"
Jul 18 01:16:53.125824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:53.125792    2567 generic.go:356] "Generic (PLEG): container finished" podID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerID="e37a3f7d01e4afbe750e846b9fc0f6ed9dc853ac288f1cc69f6096427eae2df6" exitCode=6
Jul 18 01:16:53.126224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:53.125872    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" event={"ID":"402d65fb-ab49-44be-b839-a0a8d47cb9bc","Type":"ContainerDied","Data":"e37a3f7d01e4afbe750e846b9fc0f6ed9dc853ac288f1cc69f6096427eae2df6"}
Jul 18 01:16:53.126224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:53.125933    2567 scope.go:117] "RemoveContainer" containerID="d2de8f7724ab8ded5c61401ad9a508169e04bbe623b37251de6471847def7d37"
Jul 18 01:16:53.126224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:53.126112    2567 scope.go:117] "RemoveContainer" containerID="e37a3f7d01e4afbe750e846b9fc0f6ed9dc853ac288f1cc69f6096427eae2df6"
Jul 18 01:16:53.126410 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:53.126390    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2_odh-ai-gateway-infra(402d65fb-ab49-44be-b839-a0a8d47cb9bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc"
Jul 18 01:16:54.133166 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:54.133137    2567 scope.go:117] "RemoveContainer" containerID="e37a3f7d01e4afbe750e846b9fc0f6ed9dc853ac288f1cc69f6096427eae2df6"
Jul 18 01:16:54.133570 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:16:54.133355    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2_odh-ai-gateway-infra(402d65fb-ab49-44be-b839-a0a8d47cb9bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc"
Jul 18 01:16:58.913783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:16:58.913730    2567 scope.go:117] "RemoveContainer" containerID="5f4518a8085c97ae43fcd75aec8fb941520ce69f99117294134da9707c4e2b71"
Jul 18 01:17:00.166295 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:00.166249    2567 generic.go:356] "Generic (PLEG): container finished" podID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerID="266c7fe6a7264fc9975338eca289ccc6c838c530242eb07bfe69771c6854bbae" exitCode=6
Jul 18 01:17:00.166716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:00.166324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" event={"ID":"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9","Type":"ContainerDied","Data":"266c7fe6a7264fc9975338eca289ccc6c838c530242eb07bfe69771c6854bbae"}
Jul 18 01:17:00.166716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:00.166391    2567 scope.go:117] "RemoveContainer" containerID="5f4518a8085c97ae43fcd75aec8fb941520ce69f99117294134da9707c4e2b71"
Jul 18 01:17:00.166716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:00.166693    2567 scope.go:117] "RemoveContainer" containerID="266c7fe6a7264fc9975338eca289ccc6c838c530242eb07bfe69771c6854bbae"
Jul 18 01:17:00.166969 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:00.166947    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-xmgls_odh-ai-gateway-infra(f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9"
Jul 18 01:17:01.191351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:01.191318    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"]
Jul 18 01:17:01.329174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:01.329146    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:17:01.456680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:01.456587    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9-maas-api-service-ca\") pod \"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9\" (UID: \"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9\") "
Jul 18 01:17:01.457012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:01.456986    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" (UID: "f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:17:01.557480 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:01.557442    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:17:02.180613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:02.180583    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"
Jul 18 01:17:02.180798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:02.180610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls" event={"ID":"f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9","Type":"ContainerDied","Data":"185aaa9278448ff0e42837a63a5ebd604481d793114e58820e871f39edbaf2e9"}
Jul 18 01:17:02.180798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:02.180666    2567 scope.go:117] "RemoveContainer" containerID="266c7fe6a7264fc9975338eca289ccc6c838c530242eb07bfe69771c6854bbae"
Jul 18 01:17:02.203840 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:02.203806    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"]
Jul 18 01:17:02.211212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:02.211182    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-xmgls"]
Jul 18 01:17:02.919065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:02.919032    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" path="/var/lib/kubelet/pods/f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9/volumes"
Jul 18 01:17:07.913389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:07.913358    2567 scope.go:117] "RemoveContainer" containerID="e37a3f7d01e4afbe750e846b9fc0f6ed9dc853ac288f1cc69f6096427eae2df6"
Jul 18 01:17:08.576232 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576203    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"]
Jul 18 01:17:08.576641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576626    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:08.576687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576644    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:08.576687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576665    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:08.576687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576670    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:08.576787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576722    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:08.576787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.576731    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:08.579861 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.579839    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:08.585900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.585874    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"]
Jul 18 01:17:08.727186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.727152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft\" (UID: \"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:08.827777 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.827738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft\" (UID: \"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:08.828379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.828360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft\" (UID: \"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:08.935751 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:08.935713    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:09.072099 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:09.072070    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"]
Jul 18 01:17:09.073596 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:17:09.073555    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8046ab_2894_4b49_8aa0_1645c2a7ef8c.slice/crio-fec3f6fc5fdfb0e27a8eadc1bb1d9ae9611a6ba7ddc5e55233fb9f240364fb45 WatchSource:0}: Error finding container fec3f6fc5fdfb0e27a8eadc1bb1d9ae9611a6ba7ddc5e55233fb9f240364fb45: Status 404 returned error can't find the container with id fec3f6fc5fdfb0e27a8eadc1bb1d9ae9611a6ba7ddc5e55233fb9f240364fb45
Jul 18 01:17:09.217978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:09.217938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" event={"ID":"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c","Type":"ContainerStarted","Data":"fec3f6fc5fdfb0e27a8eadc1bb1d9ae9611a6ba7ddc5e55233fb9f240364fb45"}
Jul 18 01:17:09.219750 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:09.219727    2567 generic.go:356] "Generic (PLEG): container finished" podID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerID="a2eb0dc684ff462557cd7178a8a5158aca1fdb665b4887c520d77ae913685d26" exitCode=6
Jul 18 01:17:09.219874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:09.219795    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" event={"ID":"402d65fb-ab49-44be-b839-a0a8d47cb9bc","Type":"ContainerDied","Data":"a2eb0dc684ff462557cd7178a8a5158aca1fdb665b4887c520d77ae913685d26"}
Jul 18 01:17:09.219874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:09.219843    2567 scope.go:117] "RemoveContainer" containerID="e37a3f7d01e4afbe750e846b9fc0f6ed9dc853ac288f1cc69f6096427eae2df6"
Jul 18 01:17:09.220115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:09.220099    2567 scope.go:117] "RemoveContainer" containerID="a2eb0dc684ff462557cd7178a8a5158aca1fdb665b4887c520d77ae913685d26"
Jul 18 01:17:09.220350 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:09.220327    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2_odh-ai-gateway-infra(402d65fb-ab49-44be-b839-a0a8d47cb9bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc"
Jul 18 01:17:10.226445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.226414    2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerID="b977107ea0fa9c158b6ad71d741c0265cae910d24236142e9bb74dae29ebe51a" exitCode=7
Jul 18 01:17:10.226897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.226491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" event={"ID":"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c","Type":"ContainerDied","Data":"b977107ea0fa9c158b6ad71d741c0265cae910d24236142e9bb74dae29ebe51a"}
Jul 18 01:17:10.226897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.226664    2567 scope.go:117] "RemoveContainer" containerID="b977107ea0fa9c158b6ad71d741c0265cae910d24236142e9bb74dae29ebe51a"
Jul 18 01:17:10.247024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.246993    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"]
Jul 18 01:17:10.391898 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.391873    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:17:10.545209 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.545100    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402d65fb-ab49-44be-b839-a0a8d47cb9bc-maas-api-service-ca\") pod \"402d65fb-ab49-44be-b839-a0a8d47cb9bc\" (UID: \"402d65fb-ab49-44be-b839-a0a8d47cb9bc\") "
Jul 18 01:17:10.545528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.545506    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/402d65fb-ab49-44be-b839-a0a8d47cb9bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "402d65fb-ab49-44be-b839-a0a8d47cb9bc" (UID: "402d65fb-ab49-44be-b839-a0a8d47cb9bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:17:10.646656 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:10.646622    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402d65fb-ab49-44be-b839-a0a8d47cb9bc-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:17:11.236592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.236500    2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerID="c126428a5e8c313a2c641631d5a84707a6066783659b9734a275a8928251e0d8" exitCode=7
Jul 18 01:17:11.236592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.236577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" event={"ID":"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c","Type":"ContainerDied","Data":"c126428a5e8c313a2c641631d5a84707a6066783659b9734a275a8928251e0d8"}
Jul 18 01:17:11.237108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.236656    2567 scope.go:117] "RemoveContainer" containerID="b977107ea0fa9c158b6ad71d741c0265cae910d24236142e9bb74dae29ebe51a"
Jul 18 01:17:11.237108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.236865    2567 scope.go:117] "RemoveContainer" containerID="c126428a5e8c313a2c641631d5a84707a6066783659b9734a275a8928251e0d8"
Jul 18 01:17:11.237213 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:11.237117    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft_odh-ai-gateway-infra(9f8046ab-2894-4b49-8aa0-1645c2a7ef8c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c"
Jul 18 01:17:11.237964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.237940    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"
Jul 18 01:17:11.237964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.237959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2" event={"ID":"402d65fb-ab49-44be-b839-a0a8d47cb9bc","Type":"ContainerDied","Data":"53a1b23fb879608a6387a481f17fff7fbec94f446c512811121a838245b19ec6"}
Jul 18 01:17:11.247750 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.247729    2567 scope.go:117] "RemoveContainer" containerID="a2eb0dc684ff462557cd7178a8a5158aca1fdb665b4887c520d77ae913685d26"
Jul 18 01:17:11.265989 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.265953    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"]
Jul 18 01:17:11.270036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:11.270012    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-88cj2"]
Jul 18 01:17:12.244594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:12.244564    2567 scope.go:117] "RemoveContainer" containerID="c126428a5e8c313a2c641631d5a84707a6066783659b9734a275a8928251e0d8"
Jul 18 01:17:12.245049 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:12.244793    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft_odh-ai-gateway-infra(9f8046ab-2894-4b49-8aa0-1645c2a7ef8c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c"
Jul 18 01:17:12.919972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:12.919939    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" path="/var/lib/kubelet/pods/402d65fb-ab49-44be-b839-a0a8d47cb9bc/volumes"
Jul 18 01:17:24.914057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:24.914026    2567 scope.go:117] "RemoveContainer" containerID="c126428a5e8c313a2c641631d5a84707a6066783659b9734a275a8928251e0d8"
Jul 18 01:17:25.320021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:25.319980    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" event={"ID":"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c","Type":"ContainerStarted","Data":"88cd2c2a7ae7b32f0d2612ac36644f8d9b8a05c87155cce95bd1ad4fecc19572"}
Jul 18 01:17:25.334128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:25.334076    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" podStartSLOduration=17.334062392 podStartE2EDuration="17.334062392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:17:25.333561063 +0000 UTC m=+2224.861066023" watchObservedRunningTime="2026-07-18 01:17:25.334062392 +0000 UTC m=+2224.861567351"
Jul 18 01:17:26.326856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:26.326825    2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerID="88cd2c2a7ae7b32f0d2612ac36644f8d9b8a05c87155cce95bd1ad4fecc19572" exitCode=7
Jul 18 01:17:26.327298 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:26.326897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" event={"ID":"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c","Type":"ContainerDied","Data":"88cd2c2a7ae7b32f0d2612ac36644f8d9b8a05c87155cce95bd1ad4fecc19572"}
Jul 18 01:17:26.327298 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:26.326966    2567 scope.go:117] "RemoveContainer" containerID="c126428a5e8c313a2c641631d5a84707a6066783659b9734a275a8928251e0d8"
Jul 18 01:17:26.327449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:26.327300    2567 scope.go:117] "RemoveContainer" containerID="88cd2c2a7ae7b32f0d2612ac36644f8d9b8a05c87155cce95bd1ad4fecc19572"
Jul 18 01:17:26.327537 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:26.327510    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft_odh-ai-gateway-infra(9f8046ab-2894-4b49-8aa0-1645c2a7ef8c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c"
Jul 18 01:17:26.347504 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:26.347476    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"]
Jul 18 01:17:27.472511 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:27.472483    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:27.477911 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:27.477887    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c-maas-api-service-ca\") pod \"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c\" (UID: \"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c\") "
Jul 18 01:17:27.478205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:27.478185    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" (UID: "9f8046ab-2894-4b49-8aa0-1645c2a7ef8c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:17:27.578696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:27.578663    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:17:28.340018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:28.339984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"
Jul 18 01:17:28.340018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:28.340013    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft" event={"ID":"9f8046ab-2894-4b49-8aa0-1645c2a7ef8c","Type":"ContainerDied","Data":"fec3f6fc5fdfb0e27a8eadc1bb1d9ae9611a6ba7ddc5e55233fb9f240364fb45"}
Jul 18 01:17:28.340304 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:28.340070    2567 scope.go:117] "RemoveContainer" containerID="88cd2c2a7ae7b32f0d2612ac36644f8d9b8a05c87155cce95bd1ad4fecc19572"
Jul 18 01:17:28.362605 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:28.362576    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"]
Jul 18 01:17:28.366877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:28.366855    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-s6jft"]
Jul 18 01:17:28.922066 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:28.922022    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" path="/var/lib/kubelet/pods/9f8046ab-2894-4b49-8aa0-1645c2a7ef8c/volumes"
Jul 18 01:17:33.212068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212037    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"]
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212405    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212416    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212424    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212430    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212442    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212449    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212457    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212463    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212472    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212477    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212488    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:33.212537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212493    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:33.212904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212550    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:33.212904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212557    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:33.212904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212563    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212570    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:33.212904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.212578    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4a0c09a-d6e8-42d3-8e27-0ce866d73bb9" containerName="revoke-keys"
Jul 18 01:17:33.216771 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.216754    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:33.219331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.219308    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:17:33.219470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.219335    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:17:33.223154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.223133    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"]
Jul 18 01:17:33.228621 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.228591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4080656e-d71f-454c-b223-3ec71c1b12c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-52cxq\" (UID: \"4080656e-d71f-454c-b223-3ec71c1b12c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:33.329893 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.329854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4080656e-d71f-454c-b223-3ec71c1b12c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-52cxq\" (UID: \"4080656e-d71f-454c-b223-3ec71c1b12c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:33.330508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.330489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4080656e-d71f-454c-b223-3ec71c1b12c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-52cxq\" (UID: \"4080656e-d71f-454c-b223-3ec71c1b12c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:33.529690 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.529605    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:33.876163 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:33.876135    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"]
Jul 18 01:17:33.877540 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:17:33.877515    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4080656e_d71f_454c_b223_3ec71c1b12c9.slice/crio-ef858363b3cb0e4ba808633dcaf059e3568b7fe103f3ee78cd1f5537c93e29f9 WatchSource:0}: Error finding container ef858363b3cb0e4ba808633dcaf059e3568b7fe103f3ee78cd1f5537c93e29f9: Status 404 returned error can't find the container with id ef858363b3cb0e4ba808633dcaf059e3568b7fe103f3ee78cd1f5537c93e29f9
Jul 18 01:17:34.372251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:34.372213    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerStarted","Data":"fb559368663b637f93cbc85e45a553cffdde5fbf810e7cace7ffeec7d2291f9a"}
Jul 18 01:17:34.372668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:34.372293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerStarted","Data":"ef858363b3cb0e4ba808633dcaf059e3568b7fe103f3ee78cd1f5537c93e29f9"}
Jul 18 01:17:34.390424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:34.390375    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" podStartSLOduration=1.390359233 podStartE2EDuration="1.390359233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:17:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:17:34.387988761 +0000 UTC m=+2233.915493721" watchObservedRunningTime="2026-07-18 01:17:34.390359233 +0000 UTC m=+2233.917864227"
Jul 18 01:17:35.378792 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:35.378750    2567 generic.go:356] "Generic (PLEG): container finished" podID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerID="fb559368663b637f93cbc85e45a553cffdde5fbf810e7cace7ffeec7d2291f9a" exitCode=6
Jul 18 01:17:35.381118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:35.378803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerDied","Data":"fb559368663b637f93cbc85e45a553cffdde5fbf810e7cace7ffeec7d2291f9a"}
Jul 18 01:17:35.381118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:35.379059    2567 scope.go:117] "RemoveContainer" containerID="fb559368663b637f93cbc85e45a553cffdde5fbf810e7cace7ffeec7d2291f9a"
Jul 18 01:17:36.384937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:36.384901    2567 generic.go:356] "Generic (PLEG): container finished" podID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerID="24e3b6db7bfe000f1e39189d6a6cc52dc4c2f2dfada58d5197a709efa78a94af" exitCode=6
Jul 18 01:17:36.385366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:36.384987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerDied","Data":"24e3b6db7bfe000f1e39189d6a6cc52dc4c2f2dfada58d5197a709efa78a94af"}
Jul 18 01:17:36.385366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:36.385049    2567 scope.go:117] "RemoveContainer" containerID="fb559368663b637f93cbc85e45a553cffdde5fbf810e7cace7ffeec7d2291f9a"
Jul 18 01:17:36.385366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:36.385158    2567 scope.go:117] "RemoveContainer" containerID="24e3b6db7bfe000f1e39189d6a6cc52dc4c2f2dfada58d5197a709efa78a94af"
Jul 18 01:17:36.385502 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:36.385428    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-52cxq_odh-ai-gateway-infra(4080656e-d71f-454c-b223-3ec71c1b12c9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9"
Jul 18 01:17:37.392058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:37.392030    2567 scope.go:117] "RemoveContainer" containerID="24e3b6db7bfe000f1e39189d6a6cc52dc4c2f2dfada58d5197a709efa78a94af"
Jul 18 01:17:37.392511 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:37.392240    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-52cxq_odh-ai-gateway-infra(4080656e-d71f-454c-b223-3ec71c1b12c9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9"
Jul 18 01:17:49.913557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:49.913475    2567 scope.go:117] "RemoveContainer" containerID="24e3b6db7bfe000f1e39189d6a6cc52dc4c2f2dfada58d5197a709efa78a94af"
Jul 18 01:17:50.456859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.456373    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerStarted","Data":"d0008c9b00160ef806b0f18ccd84c22c096b14ac2311e015bfdcf3a674218766"}
Jul 18 01:17:50.748392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.748312    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"]
Jul 18 01:17:50.748695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.748683    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:50.748746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.748696    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:50.748789 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.748771    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8046ab-2894-4b49-8aa0-1645c2a7ef8c" containerName="revoke-keys"
Jul 18 01:17:50.748789 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.748780    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="402d65fb-ab49-44be-b839-a0a8d47cb9bc" containerName="revoke-keys"
Jul 18 01:17:50.751769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.751753    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:17:50.758563 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.758531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"]
Jul 18 01:17:50.892962 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.892933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27867314-62f3-48f8-baf5-59f4cacbd192-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r\" (UID: \"27867314-62f3-48f8-baf5-59f4cacbd192\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:17:50.993633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.993600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27867314-62f3-48f8-baf5-59f4cacbd192-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r\" (UID: \"27867314-62f3-48f8-baf5-59f4cacbd192\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:17:50.994379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:50.994354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27867314-62f3-48f8-baf5-59f4cacbd192-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r\" (UID: \"27867314-62f3-48f8-baf5-59f4cacbd192\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:17:51.063304 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.063255    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:17:51.198555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.198531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"]
Jul 18 01:17:51.201136 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:17:51.201108    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27867314_62f3_48f8_baf5_59f4cacbd192.slice/crio-97bd43e46f55782f69baf836526cf4b5deda86032e7f45ece1bc0d314e400c8c WatchSource:0}: Error finding container 97bd43e46f55782f69baf836526cf4b5deda86032e7f45ece1bc0d314e400c8c: Status 404 returned error can't find the container with id 97bd43e46f55782f69baf836526cf4b5deda86032e7f45ece1bc0d314e400c8c
Jul 18 01:17:51.464317 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.464285    2567 generic.go:356] "Generic (PLEG): container finished" podID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerID="d0008c9b00160ef806b0f18ccd84c22c096b14ac2311e015bfdcf3a674218766" exitCode=6
Jul 18 01:17:51.464442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.464298    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerDied","Data":"d0008c9b00160ef806b0f18ccd84c22c096b14ac2311e015bfdcf3a674218766"}
Jul 18 01:17:51.464442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.464427    2567 scope.go:117] "RemoveContainer" containerID="24e3b6db7bfe000f1e39189d6a6cc52dc4c2f2dfada58d5197a709efa78a94af"
Jul 18 01:17:51.464913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.464777    2567 scope.go:117] "RemoveContainer" containerID="d0008c9b00160ef806b0f18ccd84c22c096b14ac2311e015bfdcf3a674218766"
Jul 18 01:17:51.465208 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:51.465168    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-52cxq_odh-ai-gateway-infra(4080656e-d71f-454c-b223-3ec71c1b12c9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9"
Jul 18 01:17:51.466116 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.466087    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" event={"ID":"27867314-62f3-48f8-baf5-59f4cacbd192","Type":"ContainerStarted","Data":"97bd43e46f55782f69baf836526cf4b5deda86032e7f45ece1bc0d314e400c8c"}
Jul 18 01:17:51.483888 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:51.483865    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"]
Jul 18 01:17:52.475204 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.475174    2567 generic.go:356] "Generic (PLEG): container finished" podID="27867314-62f3-48f8-baf5-59f4cacbd192" containerID="f6c618699186add16f77121c1e198be47f3ffbcd2851651dc30a1720f37a1ff3" exitCode=6
Jul 18 01:17:52.475569 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.475241    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" event={"ID":"27867314-62f3-48f8-baf5-59f4cacbd192","Type":"ContainerDied","Data":"f6c618699186add16f77121c1e198be47f3ffbcd2851651dc30a1720f37a1ff3"}
Jul 18 01:17:52.475569 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.475473    2567 scope.go:117] "RemoveContainer" containerID="f6c618699186add16f77121c1e198be47f3ffbcd2851651dc30a1720f37a1ff3"
Jul 18 01:17:52.618174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.618147    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:52.710547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.710515    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4080656e-d71f-454c-b223-3ec71c1b12c9-maas-api-service-ca\") pod \"4080656e-d71f-454c-b223-3ec71c1b12c9\" (UID: \"4080656e-d71f-454c-b223-3ec71c1b12c9\") "
Jul 18 01:17:52.710887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.710864    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4080656e-d71f-454c-b223-3ec71c1b12c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4080656e-d71f-454c-b223-3ec71c1b12c9" (UID: "4080656e-d71f-454c-b223-3ec71c1b12c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:17:52.812141 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:52.812110    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4080656e-d71f-454c-b223-3ec71c1b12c9-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:17:53.481899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.481809    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"
Jul 18 01:17:53.481899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.481815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq" event={"ID":"4080656e-d71f-454c-b223-3ec71c1b12c9","Type":"ContainerDied","Data":"ef858363b3cb0e4ba808633dcaf059e3568b7fe103f3ee78cd1f5537c93e29f9"}
Jul 18 01:17:53.481899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.481875    2567 scope.go:117] "RemoveContainer" containerID="d0008c9b00160ef806b0f18ccd84c22c096b14ac2311e015bfdcf3a674218766"
Jul 18 01:17:53.484109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.484082    2567 generic.go:356] "Generic (PLEG): container finished" podID="27867314-62f3-48f8-baf5-59f4cacbd192" containerID="2793d94c3956d10901968192970f519d20b10617f82680910e6ee1f45772f5a8" exitCode=6
Jul 18 01:17:53.484262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.484246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" event={"ID":"27867314-62f3-48f8-baf5-59f4cacbd192","Type":"ContainerDied","Data":"2793d94c3956d10901968192970f519d20b10617f82680910e6ee1f45772f5a8"}
Jul 18 01:17:53.484423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.484401    2567 scope.go:117] "RemoveContainer" containerID="2793d94c3956d10901968192970f519d20b10617f82680910e6ee1f45772f5a8"
Jul 18 01:17:53.484709 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:53.484686    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r_odh-ai-gateway-infra(27867314-62f3-48f8-baf5-59f4cacbd192)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" podUID="27867314-62f3-48f8-baf5-59f4cacbd192"
Jul 18 01:17:53.490946 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.490928    2567 scope.go:117] "RemoveContainer" containerID="f6c618699186add16f77121c1e198be47f3ffbcd2851651dc30a1720f37a1ff3"
Jul 18 01:17:53.515581 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.515545    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"]
Jul 18 01:17:53.521732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:53.521707    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-52cxq"]
Jul 18 01:17:54.491213 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:54.491177    2567 scope.go:117] "RemoveContainer" containerID="2793d94c3956d10901968192970f519d20b10617f82680910e6ee1f45772f5a8"
Jul 18 01:17:54.491706 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:17:54.491461    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r_odh-ai-gateway-infra(27867314-62f3-48f8-baf5-59f4cacbd192)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" podUID="27867314-62f3-48f8-baf5-59f4cacbd192"
Jul 18 01:17:54.919774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:54.919740    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" path="/var/lib/kubelet/pods/4080656e-d71f-454c-b223-3ec71c1b12c9/volumes"
Jul 18 01:17:58.615750 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.615712    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"]
Jul 18 01:17:58.616303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.616254    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:17:58.616303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.616295    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:17:58.616453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.616310    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:17:58.616453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.616318    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:17:58.616453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.616446    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:17:58.616602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.616462    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:17:58.619651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.619631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:17:58.626628 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.626603    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"]
Jul 18 01:17:58.764025 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.763981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548de35e-8887-4b86-b733-5ca3b57bf029-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv\" (UID: \"548de35e-8887-4b86-b733-5ca3b57bf029\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:17:58.865541 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.865504    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548de35e-8887-4b86-b733-5ca3b57bf029-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv\" (UID: \"548de35e-8887-4b86-b733-5ca3b57bf029\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:17:58.866303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.866199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548de35e-8887-4b86-b733-5ca3b57bf029-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv\" (UID: \"548de35e-8887-4b86-b733-5ca3b57bf029\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:17:58.931846 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:58.931814    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:17:59.069148 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:59.069123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"]
Jul 18 01:17:59.071076 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:17:59.071047    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548de35e_8887_4b86_b733_5ca3b57bf029.slice/crio-a655219bc0fdd221a5adaf910e6fda998b065569851e3e482805e88044b8281d WatchSource:0}: Error finding container a655219bc0fdd221a5adaf910e6fda998b065569851e3e482805e88044b8281d: Status 404 returned error can't find the container with id a655219bc0fdd221a5adaf910e6fda998b065569851e3e482805e88044b8281d
Jul 18 01:17:59.517516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:59.517414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" event={"ID":"548de35e-8887-4b86-b733-5ca3b57bf029","Type":"ContainerStarted","Data":"7046db65738b22c59fe0b3fea085bb171c96749052059287d77675a765f327e6"}
Jul 18 01:17:59.517516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:59.517474    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" event={"ID":"548de35e-8887-4b86-b733-5ca3b57bf029","Type":"ContainerStarted","Data":"a655219bc0fdd221a5adaf910e6fda998b065569851e3e482805e88044b8281d"}
Jul 18 01:17:59.533082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:17:59.533030    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" podStartSLOduration=1.5330162760000001 podStartE2EDuration="1.533016276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:17:59.531091552 +0000 UTC m=+2259.058596513" watchObservedRunningTime="2026-07-18 01:17:59.533016276 +0000 UTC m=+2259.060521236"
Jul 18 01:18:00.523523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:00.523489    2567 generic.go:356] "Generic (PLEG): container finished" podID="548de35e-8887-4b86-b733-5ca3b57bf029" containerID="7046db65738b22c59fe0b3fea085bb171c96749052059287d77675a765f327e6" exitCode=7
Jul 18 01:18:00.523917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:00.523539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" event={"ID":"548de35e-8887-4b86-b733-5ca3b57bf029","Type":"ContainerDied","Data":"7046db65738b22c59fe0b3fea085bb171c96749052059287d77675a765f327e6"}
Jul 18 01:18:00.523917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:00.523754    2567 scope.go:117] "RemoveContainer" containerID="7046db65738b22c59fe0b3fea085bb171c96749052059287d77675a765f327e6"
Jul 18 01:18:01.530655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:01.530557    2567 generic.go:356] "Generic (PLEG): container finished" podID="548de35e-8887-4b86-b733-5ca3b57bf029" containerID="f2b22e75ff84f5b1be3dc6f4256f2ee58d817b052e7a28c2973cec925e1cae63" exitCode=7
Jul 18 01:18:01.530655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:01.530626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" event={"ID":"548de35e-8887-4b86-b733-5ca3b57bf029","Type":"ContainerDied","Data":"f2b22e75ff84f5b1be3dc6f4256f2ee58d817b052e7a28c2973cec925e1cae63"}
Jul 18 01:18:01.531167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:01.530694    2567 scope.go:117] "RemoveContainer" containerID="7046db65738b22c59fe0b3fea085bb171c96749052059287d77675a765f327e6"
Jul 18 01:18:01.531167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:01.530961    2567 scope.go:117] "RemoveContainer" containerID="f2b22e75ff84f5b1be3dc6f4256f2ee58d817b052e7a28c2973cec925e1cae63"
Jul 18 01:18:01.531257 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:01.531166    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv_odh-ai-gateway-infra(548de35e-8887-4b86-b733-5ca3b57bf029)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" podUID="548de35e-8887-4b86-b733-5ca3b57bf029"
Jul 18 01:18:02.537250 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:02.537219    2567 scope.go:117] "RemoveContainer" containerID="f2b22e75ff84f5b1be3dc6f4256f2ee58d817b052e7a28c2973cec925e1cae63"
Jul 18 01:18:02.537637 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:02.537453    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv_odh-ai-gateway-infra(548de35e-8887-4b86-b733-5ca3b57bf029)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" podUID="548de35e-8887-4b86-b733-5ca3b57bf029"
Jul 18 01:18:05.913971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:05.913939    2567 scope.go:117] "RemoveContainer" containerID="2793d94c3956d10901968192970f519d20b10617f82680910e6ee1f45772f5a8"
Jul 18 01:18:06.560453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:06.560417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" event={"ID":"27867314-62f3-48f8-baf5-59f4cacbd192","Type":"ContainerStarted","Data":"e6f30b45866b1d9609ce4d10c986e0c03c2d7dcff5e2526d47dee13ecd1551cc"}
Jul 18 01:18:06.576226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:06.576172    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" podStartSLOduration=16.576156186 podStartE2EDuration="16.576156186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:18:06.57472605 +0000 UTC m=+2266.102231012" watchObservedRunningTime="2026-07-18 01:18:06.576156186 +0000 UTC m=+2266.103661147"
Jul 18 01:18:07.567739 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:07.567706    2567 generic.go:356] "Generic (PLEG): container finished" podID="27867314-62f3-48f8-baf5-59f4cacbd192" containerID="e6f30b45866b1d9609ce4d10c986e0c03c2d7dcff5e2526d47dee13ecd1551cc" exitCode=6
Jul 18 01:18:07.568156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:07.567769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" event={"ID":"27867314-62f3-48f8-baf5-59f4cacbd192","Type":"ContainerDied","Data":"e6f30b45866b1d9609ce4d10c986e0c03c2d7dcff5e2526d47dee13ecd1551cc"}
Jul 18 01:18:07.568156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:07.567818    2567 scope.go:117] "RemoveContainer" containerID="2793d94c3956d10901968192970f519d20b10617f82680910e6ee1f45772f5a8"
Jul 18 01:18:07.568156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:07.568058    2567 scope.go:117] "RemoveContainer" containerID="e6f30b45866b1d9609ce4d10c986e0c03c2d7dcff5e2526d47dee13ecd1551cc"
Jul 18 01:18:07.568332 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:07.568310    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r_odh-ai-gateway-infra(27867314-62f3-48f8-baf5-59f4cacbd192)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" podUID="27867314-62f3-48f8-baf5-59f4cacbd192"
Jul 18 01:18:07.590354 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:07.590323    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"]
Jul 18 01:18:08.711415 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:08.711378    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:18:08.856802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:08.856695    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27867314-62f3-48f8-baf5-59f4cacbd192-maas-api-service-ca\") pod \"27867314-62f3-48f8-baf5-59f4cacbd192\" (UID: \"27867314-62f3-48f8-baf5-59f4cacbd192\") "
Jul 18 01:18:08.857089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:08.857064    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27867314-62f3-48f8-baf5-59f4cacbd192-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27867314-62f3-48f8-baf5-59f4cacbd192" (UID: "27867314-62f3-48f8-baf5-59f4cacbd192"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:18:08.958285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:08.958241    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27867314-62f3-48f8-baf5-59f4cacbd192-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:09.580575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:09.580542    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r" event={"ID":"27867314-62f3-48f8-baf5-59f4cacbd192","Type":"ContainerDied","Data":"97bd43e46f55782f69baf836526cf4b5deda86032e7f45ece1bc0d314e400c8c"}
Jul 18 01:18:09.580575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:09.580579    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"
Jul 18 01:18:09.580825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:09.580604    2567 scope.go:117] "RemoveContainer" containerID="e6f30b45866b1d9609ce4d10c986e0c03c2d7dcff5e2526d47dee13ecd1551cc"
Jul 18 01:18:09.595719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:09.595692    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"]
Jul 18 01:18:09.600552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:09.600524    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-4tj6r"]
Jul 18 01:18:10.918647 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:10.918616    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" path="/var/lib/kubelet/pods/27867314-62f3-48f8-baf5-59f4cacbd192/volumes"
Jul 18 01:18:13.914082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:13.914047    2567 scope.go:117] "RemoveContainer" containerID="f2b22e75ff84f5b1be3dc6f4256f2ee58d817b052e7a28c2973cec925e1cae63"
Jul 18 01:18:14.608168 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:14.608134    2567 generic.go:356] "Generic (PLEG): container finished" podID="548de35e-8887-4b86-b733-5ca3b57bf029" containerID="1d3fedc89573049c9dd99f4d6aa7a667e5f306ff57cf7281cdbce4c20e85981d" exitCode=7
Jul 18 01:18:14.608345 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:14.608206    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" event={"ID":"548de35e-8887-4b86-b733-5ca3b57bf029","Type":"ContainerDied","Data":"1d3fedc89573049c9dd99f4d6aa7a667e5f306ff57cf7281cdbce4c20e85981d"}
Jul 18 01:18:14.608345 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:14.608323    2567 scope.go:117] "RemoveContainer" containerID="f2b22e75ff84f5b1be3dc6f4256f2ee58d817b052e7a28c2973cec925e1cae63"
Jul 18 01:18:14.608647 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:14.608621    2567 scope.go:117] "RemoveContainer" containerID="1d3fedc89573049c9dd99f4d6aa7a667e5f306ff57cf7281cdbce4c20e85981d"
Jul 18 01:18:14.608863 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:14.608842    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv_odh-ai-gateway-infra(548de35e-8887-4b86-b733-5ca3b57bf029)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" podUID="548de35e-8887-4b86-b733-5ca3b57bf029"
Jul 18 01:18:15.634035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:15.634003    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"]
Jul 18 01:18:15.776024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:15.775996    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:18:15.817693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:15.817665    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548de35e-8887-4b86-b733-5ca3b57bf029-maas-api-service-ca\") pod \"548de35e-8887-4b86-b733-5ca3b57bf029\" (UID: \"548de35e-8887-4b86-b733-5ca3b57bf029\") "
Jul 18 01:18:15.818047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:15.818025    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/548de35e-8887-4b86-b733-5ca3b57bf029-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "548de35e-8887-4b86-b733-5ca3b57bf029" (UID: "548de35e-8887-4b86-b733-5ca3b57bf029"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:18:15.918907 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:15.918817    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548de35e-8887-4b86-b733-5ca3b57bf029-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:16.621802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:16.621769    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"
Jul 18 01:18:16.621802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:16.621794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv" event={"ID":"548de35e-8887-4b86-b733-5ca3b57bf029","Type":"ContainerDied","Data":"a655219bc0fdd221a5adaf910e6fda998b065569851e3e482805e88044b8281d"}
Jul 18 01:18:16.622051 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:16.621854    2567 scope.go:117] "RemoveContainer" containerID="1d3fedc89573049c9dd99f4d6aa7a667e5f306ff57cf7281cdbce4c20e85981d"
Jul 18 01:18:16.644129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:16.644099    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"]
Jul 18 01:18:16.649515 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:16.649494    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-gsbmv"]
Jul 18 01:18:16.919770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:16.919684    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" path="/var/lib/kubelet/pods/548de35e-8887-4b86-b733-5ca3b57bf029/volumes"
Jul 18 01:18:28.590055 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:28.590015    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"]
Jul 18 01:18:28.590526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:28.590298    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy" containerID="cri-o://ce5c203368716b6b3d2ac86aa63e0fc8eca34487eb2695c20b45f88677c389fd" gracePeriod=30
Jul 18 01:18:28.647412 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:28.647379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:29.739880 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:29.739851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:31.313752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:31.313713    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:18:33.252847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.252813    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"]
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253195    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253208    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253221    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253228    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253234    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253240    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253249    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253254    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253279 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253281    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253287    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253340    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253348    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253354    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253360    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253365    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:33.253599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.253371    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4080656e-d71f-454c-b223-3ec71c1b12c9" containerName="revoke-keys"
Jul 18 01:18:33.256491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.256475    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:33.259120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.259088    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:18:33.259120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.259088    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:18:33.263355 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.263326    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"]
Jul 18 01:18:33.385738 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.385702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091b0d0b-e399-4d61-9eab-abb86ea23212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp\" (UID: \"091b0d0b-e399-4d61-9eab-abb86ea23212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:33.486561 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.486526    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091b0d0b-e399-4d61-9eab-abb86ea23212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp\" (UID: \"091b0d0b-e399-4d61-9eab-abb86ea23212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:33.487153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.487130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091b0d0b-e399-4d61-9eab-abb86ea23212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp\" (UID: \"091b0d0b-e399-4d61-9eab-abb86ea23212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:33.568933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.568896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:33.714331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.714296    2567 generic.go:356] "Generic (PLEG): container finished" podID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerID="ce5c203368716b6b3d2ac86aa63e0fc8eca34487eb2695c20b45f88677c389fd" exitCode=0
Jul 18 01:18:33.714503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.714306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" event={"ID":"3deed382-f91d-43d3-a8f6-982fd5933e71","Type":"ContainerDied","Data":"ce5c203368716b6b3d2ac86aa63e0fc8eca34487eb2695c20b45f88677c389fd"}
Jul 18 01:18:33.714503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.714440    2567 scope.go:117] "RemoveContainer" containerID="fc1fa3fa64a05955ea75756ca591f83da6fb8a3c6e9683659022e30ec009024d"
Jul 18 01:18:33.790619 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.790576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:33.850097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.850070    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:18:33.921975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.921783    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"]
Jul 18 01:18:33.924889 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:18:33.924865    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091b0d0b_e399_4d61_9eab_abb86ea23212.slice/crio-79bd18400212b17bfca18606ac606cbea00994abf773ea9dc8fee4560ce0496f WatchSource:0}: Error finding container 79bd18400212b17bfca18606ac606cbea00994abf773ea9dc8fee4560ce0496f: Status 404 returned error can't find the container with id 79bd18400212b17bfca18606ac606cbea00994abf773ea9dc8fee4560ce0496f
Jul 18 01:18:33.990433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990397    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-envoy\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990438    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-token\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990488    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-data\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990562    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fq55\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-kube-api-access-9fq55\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990588    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3deed382-f91d-43d3-a8f6-982fd5933e71-istiod-ca-cert\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990611    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-certs\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990737    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-podinfo\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990795    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-socket\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.990847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.990827    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-credential-socket\") pod \"3deed382-f91d-43d3-a8f6-982fd5933e71\" (UID: \"3deed382-f91d-43d3-a8f6-982fd5933e71\") "
Jul 18 01:18:33.991110 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.991084    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:18:33.991110 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.991082    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3deed382-f91d-43d3-a8f6-982fd5933e71-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:18:33.991412 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.991388    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:18:33.991575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.991538    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:18:33.991575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.991549    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-data" (OuterVolumeSpecName: "istio-data") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:18:33.992945 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.992925    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-token" (OuterVolumeSpecName: "istio-token") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:18:33.993200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.993178    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-kube-api-access-9fq55" (OuterVolumeSpecName: "kube-api-access-9fq55") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "kube-api-access-9fq55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:18:33.993383 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.993361    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:18:33.993383 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:33.993365    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3deed382-f91d-43d3-a8f6-982fd5933e71" (UID: "3deed382-f91d-43d3-a8f6-982fd5933e71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091837    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fq55\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-kube-api-access-9fq55\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091869    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3deed382-f91d-43d3-a8f6-982fd5933e71-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091879    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091888    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091895    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091903    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091911    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091919    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.091928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.091926    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3deed382-f91d-43d3-a8f6-982fd5933e71-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:34.517495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.517398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:34.720813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.720786    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"
Jul 18 01:18:34.720978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.720810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8" event={"ID":"3deed382-f91d-43d3-a8f6-982fd5933e71","Type":"ContainerDied","Data":"7b7989f98d4b613c7cce5e52fcddcf718f5cc576d99c3c2a9a93ab5ee513af42"}
Jul 18 01:18:34.720978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.720868    2567 scope.go:117] "RemoveContainer" containerID="ce5c203368716b6b3d2ac86aa63e0fc8eca34487eb2695c20b45f88677c389fd"
Jul 18 01:18:34.722537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.722510    2567 generic.go:356] "Generic (PLEG): container finished" podID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerID="baeda75d743f0f54bc9d8c2b592c5dd7c25b3c5449705b73bb7a35677cef9f15" exitCode=6
Jul 18 01:18:34.722633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.722568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" event={"ID":"091b0d0b-e399-4d61-9eab-abb86ea23212","Type":"ContainerDied","Data":"baeda75d743f0f54bc9d8c2b592c5dd7c25b3c5449705b73bb7a35677cef9f15"}
Jul 18 01:18:34.722633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.722612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" event={"ID":"091b0d0b-e399-4d61-9eab-abb86ea23212","Type":"ContainerStarted","Data":"79bd18400212b17bfca18606ac606cbea00994abf773ea9dc8fee4560ce0496f"}
Jul 18 01:18:34.722794 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.722781    2567 scope.go:117] "RemoveContainer" containerID="baeda75d743f0f54bc9d8c2b592c5dd7c25b3c5449705b73bb7a35677cef9f15"
Jul 18 01:18:34.750329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.750298    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"]
Jul 18 01:18:34.754995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.754964    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d5466ad4-openshift-default-566948f57f-fwwm8"]
Jul 18 01:18:34.919972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:34.919938    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" path="/var/lib/kubelet/pods/3deed382-f91d-43d3-a8f6-982fd5933e71/volumes"
Jul 18 01:18:35.730202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:35.730109    2567 generic.go:356] "Generic (PLEG): container finished" podID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerID="790fc520304afddc353d35724927ca4413fb34f575f5963280cb5e1aaa3227f0" exitCode=6
Jul 18 01:18:35.730699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:35.730199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" event={"ID":"091b0d0b-e399-4d61-9eab-abb86ea23212","Type":"ContainerDied","Data":"790fc520304afddc353d35724927ca4413fb34f575f5963280cb5e1aaa3227f0"}
Jul 18 01:18:35.730699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:35.730293    2567 scope.go:117] "RemoveContainer" containerID="baeda75d743f0f54bc9d8c2b592c5dd7c25b3c5449705b73bb7a35677cef9f15"
Jul 18 01:18:35.730699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:35.730488    2567 scope.go:117] "RemoveContainer" containerID="790fc520304afddc353d35724927ca4413fb34f575f5963280cb5e1aaa3227f0"
Jul 18 01:18:35.730878 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:35.730748    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp_odh-ai-gateway-infra(091b0d0b-e399-4d61-9eab-abb86ea23212)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212"
Jul 18 01:18:36.739748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:36.739723    2567 scope.go:117] "RemoveContainer" containerID="790fc520304afddc353d35724927ca4413fb34f575f5963280cb5e1aaa3227f0"
Jul 18 01:18:36.740207 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:36.739914    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp_odh-ai-gateway-infra(091b0d0b-e399-4d61-9eab-abb86ea23212)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212"
Jul 18 01:18:40.051770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.051737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:40.109934 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.109900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:40.276402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276372    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"]
Jul 18 01:18:40.276783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276770    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy"
Jul 18 01:18:40.276783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276783    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276794    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276800    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="548de35e-8887-4b86-b733-5ca3b57bf029" containerName="revoke-keys"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276813    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276818    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276826    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276832    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276892    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27867314-62f3-48f8-baf5-59f4cacbd192" containerName="revoke-keys"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276901    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy"
Jul 18 01:18:40.277008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.276909    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3deed382-f91d-43d3-a8f6-982fd5933e71" containerName="istio-proxy"
Jul 18 01:18:40.281330 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.281304    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.284823 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.284798    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8942d9e2-openshift-default-dockercfg-glq9w\""
Jul 18 01:18:40.293967 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.293941    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"]
Jul 18 01:18:40.318032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.317998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:40.450363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450299    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-token\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-data\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-credential-socket\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450465    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-podinfo\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-certs\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450749 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450578    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istiod-ca-cert\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450749 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-envoy\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450749 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450629    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-socket\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.450749 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.450653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97hdg\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-kube-api-access-97hdg\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.551784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.551743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-token\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.551784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.551793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-data\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.551839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-credential-socket\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.551864    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-podinfo\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.551975    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-certs\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552028    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istiod-ca-cert\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552336 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552070    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-envoy\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552336 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-socket\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552336 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97hdg\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-kube-api-access-97hdg\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552515 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-certs\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552515 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552411    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-socket\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552581    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-data\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552681 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552646    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-credential-socket\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.552868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.552848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istiod-ca-cert\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.554617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.554593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-envoy\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.554982 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.554962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-podinfo\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.559654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.559626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-token\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.559740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.559728    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97hdg\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-kube-api-access-97hdg\") pod \"e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") " pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.595052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.594947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:40.746130 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.746106    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"]
Jul 18 01:18:40.748454 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:18:40.748412    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e2afd5_8888_48a8_aa8f_68136a737e2f.slice/crio-7a4b30d6d5b7839c177c7b3914f59a7fee18cbf6057e07ed7b3713ba4df4f95c WatchSource:0}: Error finding container 7a4b30d6d5b7839c177c7b3914f59a7fee18cbf6057e07ed7b3713ba4df4f95c: Status 404 returned error can't find the container with id 7a4b30d6d5b7839c177c7b3914f59a7fee18cbf6057e07ed7b3713ba4df4f95c
Jul 18 01:18:40.750781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.750742    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:18:40.750886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.750813    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:18:40.750886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.750848    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:18:40.760614 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.760590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" event={"ID":"d0e2afd5-8888-48a8-aa8f-68136a737e2f","Type":"ContainerStarted","Data":"7a4b30d6d5b7839c177c7b3914f59a7fee18cbf6057e07ed7b3713ba4df4f95c"}
Jul 18 01:18:40.785817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.785787    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"]
Jul 18 01:18:40.790392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.790371    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:18:40.796217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.796105    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"]
Jul 18 01:18:40.957042 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:40.956938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0d192f-f3a0-47be-ae52-596b5a160183-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw\" (UID: \"de0d192f-f3a0-47be-ae52-596b5a160183\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:18:41.057413 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.057378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0d192f-f3a0-47be-ae52-596b5a160183-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw\" (UID: \"de0d192f-f3a0-47be-ae52-596b5a160183\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:18:41.058020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.058002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0d192f-f3a0-47be-ae52-596b5a160183-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw\" (UID: \"de0d192f-f3a0-47be-ae52-596b5a160183\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:18:41.114108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.114076    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:18:41.259701 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.259676    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"]
Jul 18 01:18:41.262363 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:18:41.262328    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0d192f_f3a0_47be_ae52_596b5a160183.slice/crio-f502cd2fed0bc3d50a44c60c2c7f8e2d1806a6e2a808821913aaa61abdc6da25 WatchSource:0}: Error finding container f502cd2fed0bc3d50a44c60c2c7f8e2d1806a6e2a808821913aaa61abdc6da25: Status 404 returned error can't find the container with id f502cd2fed0bc3d50a44c60c2c7f8e2d1806a6e2a808821913aaa61abdc6da25
Jul 18 01:18:41.444024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.443989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:41.531974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.531877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:41.608178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.608140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:41.774901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.774858    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" event={"ID":"d0e2afd5-8888-48a8-aa8f-68136a737e2f","Type":"ContainerStarted","Data":"94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"}
Jul 18 01:18:41.776436 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.776412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" event={"ID":"de0d192f-f3a0-47be-ae52-596b5a160183","Type":"ContainerStarted","Data":"952a47cdd2aeec31c10003560c3a569ae223c0c7a19842899019d8b41136aa2f"}
Jul 18 01:18:41.776532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.776454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" event={"ID":"de0d192f-f3a0-47be-ae52-596b5a160183","Type":"ContainerStarted","Data":"f502cd2fed0bc3d50a44c60c2c7f8e2d1806a6e2a808821913aaa61abdc6da25"}
Jul 18 01:18:41.799534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.799434    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" podStartSLOduration=1.799417592 podStartE2EDuration="1.799417592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:18:41.797526194 +0000 UTC m=+2301.325031179" watchObservedRunningTime="2026-07-18 01:18:41.799417592 +0000 UTC m=+2301.326922560"
Jul 18 01:18:41.818041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.817988    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" podStartSLOduration=1.817973291 podStartE2EDuration="1.817973291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:18:41.813428228 +0000 UTC m=+2301.340933187" watchObservedRunningTime="2026-07-18 01:18:41.817973291 +0000 UTC m=+2301.345478250"
Jul 18 01:18:41.947860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:41.947818    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:42.118909 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.118870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:42.218151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.218118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:42.595897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.595859    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:42.601247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.601220    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:42.782783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.782750    2567 generic.go:356] "Generic (PLEG): container finished" podID="de0d192f-f3a0-47be-ae52-596b5a160183" containerID="952a47cdd2aeec31c10003560c3a569ae223c0c7a19842899019d8b41136aa2f" exitCode=6
Jul 18 01:18:42.782938 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.782844    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" event={"ID":"de0d192f-f3a0-47be-ae52-596b5a160183","Type":"ContainerDied","Data":"952a47cdd2aeec31c10003560c3a569ae223c0c7a19842899019d8b41136aa2f"}
Jul 18 01:18:42.783044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.783030    2567 scope.go:117] "RemoveContainer" containerID="952a47cdd2aeec31c10003560c3a569ae223c0c7a19842899019d8b41136aa2f"
Jul 18 01:18:42.783122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.783095    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:42.784348 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:42.784331    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:43.790556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:43.790472    2567 generic.go:356] "Generic (PLEG): container finished" podID="de0d192f-f3a0-47be-ae52-596b5a160183" containerID="f795f316c06d27832a78943066aab8fe3f1c11e5830bd258d7325f7091f1da16" exitCode=6
Jul 18 01:18:43.790556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:43.790525    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" event={"ID":"de0d192f-f3a0-47be-ae52-596b5a160183","Type":"ContainerDied","Data":"f795f316c06d27832a78943066aab8fe3f1c11e5830bd258d7325f7091f1da16"}
Jul 18 01:18:43.791016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:43.790605    2567 scope.go:117] "RemoveContainer" containerID="952a47cdd2aeec31c10003560c3a569ae223c0c7a19842899019d8b41136aa2f"
Jul 18 01:18:43.791016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:43.790821    2567 scope.go:117] "RemoveContainer" containerID="f795f316c06d27832a78943066aab8fe3f1c11e5830bd258d7325f7091f1da16"
Jul 18 01:18:43.791095 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:43.791053    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw_odh-ai-gateway-infra(de0d192f-f3a0-47be-ae52-596b5a160183)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" podUID="de0d192f-f3a0-47be-ae52-596b5a160183"
Jul 18 01:18:44.507657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:44.507613    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:18:44.797508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:44.797420    2567 scope.go:117] "RemoveContainer" containerID="f795f316c06d27832a78943066aab8fe3f1c11e5830bd258d7325f7091f1da16"
Jul 18 01:18:44.797878 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:44.797600    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw_odh-ai-gateway-infra(de0d192f-f3a0-47be-ae52-596b5a160183)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" podUID="de0d192f-f3a0-47be-ae52-596b5a160183"
Jul 18 01:18:45.851111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:45.851067    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"]
Jul 18 01:18:45.856736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:45.856702    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:18:45.861233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:45.861201    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"]
Jul 18 01:18:46.006507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.006466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6811a73e-7b56-460b-89ee-7cef815d3f93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7\" (UID: \"6811a73e-7b56-460b-89ee-7cef815d3f93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:18:46.108019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.107916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6811a73e-7b56-460b-89ee-7cef815d3f93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7\" (UID: \"6811a73e-7b56-460b-89ee-7cef815d3f93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:18:46.108616 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.108593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6811a73e-7b56-460b-89ee-7cef815d3f93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7\" (UID: \"6811a73e-7b56-460b-89ee-7cef815d3f93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:18:46.174985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.174930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:18:46.346716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.346644    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"]
Jul 18 01:18:46.349949 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:18:46.349923    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6811a73e_7b56_460b_89ee_7cef815d3f93.slice/crio-92efba566518b13307f95d8d820611fb3f2d20633a7386df9ed019cdff38cfe4 WatchSource:0}: Error finding container 92efba566518b13307f95d8d820611fb3f2d20633a7386df9ed019cdff38cfe4: Status 404 returned error can't find the container with id 92efba566518b13307f95d8d820611fb3f2d20633a7386df9ed019cdff38cfe4
Jul 18 01:18:46.809231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.809197    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" event={"ID":"6811a73e-7b56-460b-89ee-7cef815d3f93","Type":"ContainerStarted","Data":"bdd8976b9c18268b93ccf04453df8f9401ef602e27e2a7d65a3d00f509719af5"}
Jul 18 01:18:46.809432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.809246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" event={"ID":"6811a73e-7b56-460b-89ee-7cef815d3f93","Type":"ContainerStarted","Data":"92efba566518b13307f95d8d820611fb3f2d20633a7386df9ed019cdff38cfe4"}
Jul 18 01:18:46.823087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:46.823036    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" podStartSLOduration=1.823020539 podStartE2EDuration="1.823020539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:18:46.822781756 +0000 UTC m=+2306.350286711" watchObservedRunningTime="2026-07-18 01:18:46.823020539 +0000 UTC m=+2306.350525499"
Jul 18 01:18:47.815429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:47.815395    2567 generic.go:356] "Generic (PLEG): container finished" podID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerID="bdd8976b9c18268b93ccf04453df8f9401ef602e27e2a7d65a3d00f509719af5" exitCode=7
Jul 18 01:18:47.816005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:47.815435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" event={"ID":"6811a73e-7b56-460b-89ee-7cef815d3f93","Type":"ContainerDied","Data":"bdd8976b9c18268b93ccf04453df8f9401ef602e27e2a7d65a3d00f509719af5"}
Jul 18 01:18:47.816085 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:47.815686    2567 scope.go:117] "RemoveContainer" containerID="bdd8976b9c18268b93ccf04453df8f9401ef602e27e2a7d65a3d00f509719af5"
Jul 18 01:18:47.913900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:47.913537    2567 scope.go:117] "RemoveContainer" containerID="790fc520304afddc353d35724927ca4413fb34f575f5963280cb5e1aaa3227f0"
Jul 18 01:18:48.657828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.657788    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"]
Jul 18 01:18:48.662395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.662368    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:18:48.672430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.672399    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"]
Jul 18 01:18:48.823747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.823715    2567 generic.go:356] "Generic (PLEG): container finished" podID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerID="929bb089b9798e98956de345cd3318c496a810fc36124d9a11c1a775a7854def" exitCode=6
Jul 18 01:18:48.824149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.823786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" event={"ID":"091b0d0b-e399-4d61-9eab-abb86ea23212","Type":"ContainerDied","Data":"929bb089b9798e98956de345cd3318c496a810fc36124d9a11c1a775a7854def"}
Jul 18 01:18:48.824149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.823846    2567 scope.go:117] "RemoveContainer" containerID="790fc520304afddc353d35724927ca4413fb34f575f5963280cb5e1aaa3227f0"
Jul 18 01:18:48.824149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.824080    2567 scope.go:117] "RemoveContainer" containerID="929bb089b9798e98956de345cd3318c496a810fc36124d9a11c1a775a7854def"
Jul 18 01:18:48.824329 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:48.824297    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp_odh-ai-gateway-infra(091b0d0b-e399-4d61-9eab-abb86ea23212)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212"
Jul 18 01:18:48.825954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.825935    2567 generic.go:356] "Generic (PLEG): container finished" podID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerID="9860c550e2db84b36417262f7c310fcf579108858308b84a62e8a3e3a62cc076" exitCode=7
Jul 18 01:18:48.826046 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.825989    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" event={"ID":"6811a73e-7b56-460b-89ee-7cef815d3f93","Type":"ContainerDied","Data":"9860c550e2db84b36417262f7c310fcf579108858308b84a62e8a3e3a62cc076"}
Jul 18 01:18:48.826216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.826203    2567 scope.go:117] "RemoveContainer" containerID="9860c550e2db84b36417262f7c310fcf579108858308b84a62e8a3e3a62cc076"
Jul 18 01:18:48.826398 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:48.826383    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7_odh-ai-gateway-infra(6811a73e-7b56-460b-89ee-7cef815d3f93)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93"
Jul 18 01:18:48.831553 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.831534    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f41494-a2d0-46d6-8217-b1e7c104c2d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf\" (UID: \"44f41494-a2d0-46d6-8217-b1e7c104c2d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:18:48.880397 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.880341    2567 scope.go:117] "RemoveContainer" containerID="bdd8976b9c18268b93ccf04453df8f9401ef602e27e2a7d65a3d00f509719af5"
Jul 18 01:18:48.932200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.932057    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f41494-a2d0-46d6-8217-b1e7c104c2d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf\" (UID: \"44f41494-a2d0-46d6-8217-b1e7c104c2d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:18:48.932695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.932669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f41494-a2d0-46d6-8217-b1e7c104c2d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf\" (UID: \"44f41494-a2d0-46d6-8217-b1e7c104c2d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:18:48.992959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:48.992926    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:18:49.180297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:49.163010    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"]
Jul 18 01:18:49.181403 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:18:49.181354    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f41494_a2d0_46d6_8217_b1e7c104c2d2.slice/crio-bccc39312a4a338debdb439f9731395a8975d0760f93c752dd694bf9c44f0a78 WatchSource:0}: Error finding container bccc39312a4a338debdb439f9731395a8975d0760f93c752dd694bf9c44f0a78: Status 404 returned error can't find the container with id bccc39312a4a338debdb439f9731395a8975d0760f93c752dd694bf9c44f0a78
Jul 18 01:18:49.836399 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:49.836305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" event={"ID":"44f41494-a2d0-46d6-8217-b1e7c104c2d2","Type":"ContainerStarted","Data":"5349f66e3c27608dd4c64a43a2dc2ad3583c4880550a856fc7ccca9aa251769e"}
Jul 18 01:18:49.836399 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:49.836365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" event={"ID":"44f41494-a2d0-46d6-8217-b1e7c104c2d2","Type":"ContainerStarted","Data":"bccc39312a4a338debdb439f9731395a8975d0760f93c752dd694bf9c44f0a78"}
Jul 18 01:18:49.838219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:49.838195    2567 scope.go:117] "RemoveContainer" containerID="9860c550e2db84b36417262f7c310fcf579108858308b84a62e8a3e3a62cc076"
Jul 18 01:18:49.838484 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:49.838459    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7_odh-ai-gateway-infra(6811a73e-7b56-460b-89ee-7cef815d3f93)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93"
Jul 18 01:18:49.846413 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:49.846388    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"]
Jul 18 01:18:49.853613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:49.853515    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" podStartSLOduration=1.853494695 podStartE2EDuration="1.853494695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:18:49.851039231 +0000 UTC m=+2309.378544212" watchObservedRunningTime="2026-07-18 01:18:49.853494695 +0000 UTC m=+2309.380999656"
Jul 18 01:18:50.044353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.044328    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:50.142026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.141939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091b0d0b-e399-4d61-9eab-abb86ea23212-maas-api-service-ca\") pod \"091b0d0b-e399-4d61-9eab-abb86ea23212\" (UID: \"091b0d0b-e399-4d61-9eab-abb86ea23212\") "
Jul 18 01:18:50.142342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.142319    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091b0d0b-e399-4d61-9eab-abb86ea23212-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "091b0d0b-e399-4d61-9eab-abb86ea23212" (UID: "091b0d0b-e399-4d61-9eab-abb86ea23212"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:18:50.243627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.243592    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091b0d0b-e399-4d61-9eab-abb86ea23212-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:18:50.847740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.847707    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"
Jul 18 01:18:50.847740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.847730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp" event={"ID":"091b0d0b-e399-4d61-9eab-abb86ea23212","Type":"ContainerDied","Data":"79bd18400212b17bfca18606ac606cbea00994abf773ea9dc8fee4560ce0496f"}
Jul 18 01:18:50.848295 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.847803    2567 scope.go:117] "RemoveContainer" containerID="929bb089b9798e98956de345cd3318c496a810fc36124d9a11c1a775a7854def"
Jul 18 01:18:50.849987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.849822    2567 generic.go:356] "Generic (PLEG): container finished" podID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerID="5349f66e3c27608dd4c64a43a2dc2ad3583c4880550a856fc7ccca9aa251769e" exitCode=6
Jul 18 01:18:50.849987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.849933    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" event={"ID":"44f41494-a2d0-46d6-8217-b1e7c104c2d2","Type":"ContainerDied","Data":"5349f66e3c27608dd4c64a43a2dc2ad3583c4880550a856fc7ccca9aa251769e"}
Jul 18 01:18:50.850177 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.850129    2567 scope.go:117] "RemoveContainer" containerID="5349f66e3c27608dd4c64a43a2dc2ad3583c4880550a856fc7ccca9aa251769e"
Jul 18 01:18:50.882238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.882204    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"]
Jul 18 01:18:50.887346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.887309    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-sx4bp"]
Jul 18 01:18:50.922663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.922525    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:50.932244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.932209    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" path="/var/lib/kubelet/pods/091b0d0b-e399-4d61-9eab-abb86ea23212/volumes"
Jul 18 01:18:50.975928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:50.975890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:51.859212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:51.859170    2567 generic.go:356] "Generic (PLEG): container finished" podID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerID="0d3702677624b67b8f7bb2fdcd27630e3cd99461e5bffa75d7350244ad332562" exitCode=6
Jul 18 01:18:51.859729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:51.859256    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" event={"ID":"44f41494-a2d0-46d6-8217-b1e7c104c2d2","Type":"ContainerDied","Data":"0d3702677624b67b8f7bb2fdcd27630e3cd99461e5bffa75d7350244ad332562"}
Jul 18 01:18:51.859729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:51.859380    2567 scope.go:117] "RemoveContainer" containerID="5349f66e3c27608dd4c64a43a2dc2ad3583c4880550a856fc7ccca9aa251769e"
Jul 18 01:18:51.859729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:51.859477    2567 scope.go:117] "RemoveContainer" containerID="0d3702677624b67b8f7bb2fdcd27630e3cd99461e5bffa75d7350244ad332562"
Jul 18 01:18:51.859888 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:51.859765    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf_odh-ai-gateway-infra(44f41494-a2d0-46d6-8217-b1e7c104c2d2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2"
Jul 18 01:18:52.866321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.866214    2567 scope.go:117] "RemoveContainer" containerID="0d3702677624b67b8f7bb2fdcd27630e3cd99461e5bffa75d7350244ad332562"
Jul 18 01:18:52.866789 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:18:52.866468    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf_odh-ai-gateway-infra(44f41494-a2d0-46d6-8217-b1e7c104c2d2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2"
Jul 18 01:18:52.867545 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.867520    2567 generic.go:356] "Generic (PLEG): container finished" podID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerID="94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d" exitCode=0
Jul 18 01:18:52.867652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.867590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" event={"ID":"d0e2afd5-8888-48a8-aa8f-68136a737e2f","Type":"ContainerDied","Data":"94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"}
Jul 18 01:18:52.867934 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.867921    2567 scope.go:117] "RemoveContainer" containerID="94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"
Jul 18 01:18:52.868862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.868835    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:18:52.868922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.868908    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:18:52.868957 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:52.868943    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:18:53.875588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:53.875548    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" event={"ID":"d0e2afd5-8888-48a8-aa8f-68136a737e2f","Type":"ContainerStarted","Data":"f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d"}
Jul 18 01:18:54.435689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:54.435648    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:18:54.595189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:54.595144    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:54.596503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:54.596477    2567 patch_prober.go:28] interesting pod/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 18 01:18:54.596616 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:54.596534    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 18 01:18:55.595712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:55.595677    2567 patch_prober.go:28] interesting pod/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 18 01:18:55.596113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:55.595738    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 18 01:18:56.595492 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:56.595459    2567 patch_prober.go:28] interesting pod/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 18 01:18:56.595683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:56.595516    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 18 01:18:57.595522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:57.595477    2567 patch_prober.go:28] interesting pod/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 18 01:18:57.596015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:57.595546    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 18 01:18:57.783191 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:57.783153    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:58.599442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:58.599410    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:58.600519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:58.600499    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:18:59.913626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:59.913597    2567 scope.go:117] "RemoveContainer" containerID="f795f316c06d27832a78943066aab8fe3f1c11e5830bd258d7325f7091f1da16"
Jul 18 01:18:59.914130 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:18:59.913663    2567 scope.go:117] "RemoveContainer" containerID="9860c550e2db84b36417262f7c310fcf579108858308b84a62e8a3e3a62cc076"
Jul 18 01:19:00.912259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.912176    2567 generic.go:356] "Generic (PLEG): container finished" podID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerID="757a5b637dfc5f4f8c1f004facb66c3e0ede984eb05e531f2cfa2d996aab6a36" exitCode=7
Jul 18 01:19:00.912259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.912245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" event={"ID":"6811a73e-7b56-460b-89ee-7cef815d3f93","Type":"ContainerDied","Data":"757a5b637dfc5f4f8c1f004facb66c3e0ede984eb05e531f2cfa2d996aab6a36"}
Jul 18 01:19:00.912538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.912328    2567 scope.go:117] "RemoveContainer" containerID="9860c550e2db84b36417262f7c310fcf579108858308b84a62e8a3e3a62cc076"
Jul 18 01:19:00.912761 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.912649    2567 scope.go:117] "RemoveContainer" containerID="757a5b637dfc5f4f8c1f004facb66c3e0ede984eb05e531f2cfa2d996aab6a36"
Jul 18 01:19:00.912927 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:00.912904    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7_odh-ai-gateway-infra(6811a73e-7b56-460b-89ee-7cef815d3f93)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93"
Jul 18 01:19:00.914444 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.914419    2567 generic.go:356] "Generic (PLEG): container finished" podID="de0d192f-f3a0-47be-ae52-596b5a160183" containerID="20b56dd6dc3f1ab07630258353e49b5975ff012bd84f6034f1dc5e75b70f26d1" exitCode=6
Jul 18 01:19:00.919042 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.919020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" event={"ID":"de0d192f-f3a0-47be-ae52-596b5a160183","Type":"ContainerDied","Data":"20b56dd6dc3f1ab07630258353e49b5975ff012bd84f6034f1dc5e75b70f26d1"}
Jul 18 01:19:00.919346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.919332    2567 scope.go:117] "RemoveContainer" containerID="20b56dd6dc3f1ab07630258353e49b5975ff012bd84f6034f1dc5e75b70f26d1"
Jul 18 01:19:00.919533 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:00.919516    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw_odh-ai-gateway-infra(de0d192f-f3a0-47be-ae52-596b5a160183)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" podUID="de0d192f-f3a0-47be-ae52-596b5a160183"
Jul 18 01:19:00.924439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:00.924234    2567 scope.go:117] "RemoveContainer" containerID="f795f316c06d27832a78943066aab8fe3f1c11e5830bd258d7325f7091f1da16"
Jul 18 01:19:01.937555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:01.937519    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"]
Jul 18 01:19:01.948708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:01.948671    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"]
Jul 18 01:19:02.102723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.102693    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:19:02.137139 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.137111    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:19:02.146774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.146741    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6811a73e-7b56-460b-89ee-7cef815d3f93-maas-api-service-ca\") pod \"6811a73e-7b56-460b-89ee-7cef815d3f93\" (UID: \"6811a73e-7b56-460b-89ee-7cef815d3f93\") "
Jul 18 01:19:02.147119 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.147099    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6811a73e-7b56-460b-89ee-7cef815d3f93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6811a73e-7b56-460b-89ee-7cef815d3f93" (UID: "6811a73e-7b56-460b-89ee-7cef815d3f93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:02.247930 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.247840    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0d192f-f3a0-47be-ae52-596b5a160183-maas-api-service-ca\") pod \"de0d192f-f3a0-47be-ae52-596b5a160183\" (UID: \"de0d192f-f3a0-47be-ae52-596b5a160183\") "
Jul 18 01:19:02.248089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.248009    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6811a73e-7b56-460b-89ee-7cef815d3f93-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:19:02.248224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.248199    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de0d192f-f3a0-47be-ae52-596b5a160183-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de0d192f-f3a0-47be-ae52-596b5a160183" (UID: "de0d192f-f3a0-47be-ae52-596b5a160183"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:02.348625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.348587    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0d192f-f3a0-47be-ae52-596b5a160183-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:19:02.928332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.928302    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"
Jul 18 01:19:02.928332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.928311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw" event={"ID":"de0d192f-f3a0-47be-ae52-596b5a160183","Type":"ContainerDied","Data":"f502cd2fed0bc3d50a44c60c2c7f8e2d1806a6e2a808821913aaa61abdc6da25"}
Jul 18 01:19:02.928554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.928370    2567 scope.go:117] "RemoveContainer" containerID="20b56dd6dc3f1ab07630258353e49b5975ff012bd84f6034f1dc5e75b70f26d1"
Jul 18 01:19:02.930175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.930111    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7" event={"ID":"6811a73e-7b56-460b-89ee-7cef815d3f93","Type":"ContainerDied","Data":"92efba566518b13307f95d8d820611fb3f2d20633a7386df9ed019cdff38cfe4"}
Jul 18 01:19:02.930293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.930200    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"
Jul 18 01:19:02.938633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.938476    2567 scope.go:117] "RemoveContainer" containerID="757a5b637dfc5f4f8c1f004facb66c3e0ede984eb05e531f2cfa2d996aab6a36"
Jul 18 01:19:02.982932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:02.982891    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"]
Jul 18 01:19:03.006380 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:03.006348    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-hs9lw"]
Jul 18 01:19:03.028382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:03.028357    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"]
Jul 18 01:19:03.038574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:03.038546    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-w7ht7"]
Jul 18 01:19:04.914263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:04.914229    2567 scope.go:117] "RemoveContainer" containerID="0d3702677624b67b8f7bb2fdcd27630e3cd99461e5bffa75d7350244ad332562"
Jul 18 01:19:04.921032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:04.921007    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" path="/var/lib/kubelet/pods/6811a73e-7b56-460b-89ee-7cef815d3f93/volumes"
Jul 18 01:19:04.921482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:04.921459    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" path="/var/lib/kubelet/pods/de0d192f-f3a0-47be-ae52-596b5a160183/volumes"
Jul 18 01:19:05.948929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:05.948844    2567 generic.go:356] "Generic (PLEG): container finished" podID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerID="143717b5879d4d2eb7add6fe8473c9abd3d9b3b87a13c6f23f33bad1f9fa7791" exitCode=6
Jul 18 01:19:05.949364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:05.948927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" event={"ID":"44f41494-a2d0-46d6-8217-b1e7c104c2d2","Type":"ContainerDied","Data":"143717b5879d4d2eb7add6fe8473c9abd3d9b3b87a13c6f23f33bad1f9fa7791"}
Jul 18 01:19:05.949364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:05.948993    2567 scope.go:117] "RemoveContainer" containerID="0d3702677624b67b8f7bb2fdcd27630e3cd99461e5bffa75d7350244ad332562"
Jul 18 01:19:05.949364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:05.949337    2567 scope.go:117] "RemoveContainer" containerID="143717b5879d4d2eb7add6fe8473c9abd3d9b3b87a13c6f23f33bad1f9fa7791"
Jul 18 01:19:05.949571 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:05.949549    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf_odh-ai-gateway-infra(44f41494-a2d0-46d6-8217-b1e7c104c2d2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2"
Jul 18 01:19:06.971347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:06.971312    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"]
Jul 18 01:19:07.111356 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.111332    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:19:07.191942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.191907    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f41494-a2d0-46d6-8217-b1e7c104c2d2-maas-api-service-ca\") pod \"44f41494-a2d0-46d6-8217-b1e7c104c2d2\" (UID: \"44f41494-a2d0-46d6-8217-b1e7c104c2d2\") "
Jul 18 01:19:07.192311 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.192256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44f41494-a2d0-46d6-8217-b1e7c104c2d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44f41494-a2d0-46d6-8217-b1e7c104c2d2" (UID: "44f41494-a2d0-46d6-8217-b1e7c104c2d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:07.299075 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.293150    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f41494-a2d0-46d6-8217-b1e7c104c2d2-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:19:07.961641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.961610    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"
Jul 18 01:19:07.961937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.961612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf" event={"ID":"44f41494-a2d0-46d6-8217-b1e7c104c2d2","Type":"ContainerDied","Data":"bccc39312a4a338debdb439f9731395a8975d0760f93c752dd694bf9c44f0a78"}
Jul 18 01:19:07.961937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.961745    2567 scope.go:117] "RemoveContainer" containerID="143717b5879d4d2eb7add6fe8473c9abd3d9b3b87a13c6f23f33bad1f9fa7791"
Jul 18 01:19:07.982896 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.982858    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"]
Jul 18 01:19:07.985324 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:07.985303    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p2mbf"]
Jul 18 01:19:08.919732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:08.919699    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" path="/var/lib/kubelet/pods/44f41494-a2d0-46d6-8217-b1e7c104c2d2/volumes"
Jul 18 01:19:23.290151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.290114    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"]
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291081    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291111    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291137    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291146    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291168    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291185    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291204    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291213    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291223    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291232    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291252    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291261    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291306    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291325    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291338    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291346    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291589    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291606    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291615    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291624    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291632    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291647    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291656    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291664    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291672    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:23.293449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.291681    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:23.298214 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.298174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:23.301263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.301240    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:19:23.301498 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.301329    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:19:23.302641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.302617    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"]
Jul 18 01:19:23.442065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.442029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a58eaf-18a2-4499-9520-d39565c4e2b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-j466s\" (UID: \"48a58eaf-18a2-4499-9520-d39565c4e2b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:23.542877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.542769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a58eaf-18a2-4499-9520-d39565c4e2b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-j466s\" (UID: \"48a58eaf-18a2-4499-9520-d39565c4e2b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:23.543411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.543389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a58eaf-18a2-4499-9520-d39565c4e2b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-j466s\" (UID: \"48a58eaf-18a2-4499-9520-d39565c4e2b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:23.611722 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.611688    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:23.749963 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:23.749937    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"]
Jul 18 01:19:23.752545 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:19:23.752496    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a58eaf_18a2_4499_9520_d39565c4e2b5.slice/crio-ed97aac742ed38571f6e94b6e0583399b32fbd2ecfe9c28765a652188f202a54 WatchSource:0}: Error finding container ed97aac742ed38571f6e94b6e0583399b32fbd2ecfe9c28765a652188f202a54: Status 404 returned error can't find the container with id ed97aac742ed38571f6e94b6e0583399b32fbd2ecfe9c28765a652188f202a54
Jul 18 01:19:24.045565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:24.045527    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" event={"ID":"48a58eaf-18a2-4499-9520-d39565c4e2b5","Type":"ContainerStarted","Data":"ed97aac742ed38571f6e94b6e0583399b32fbd2ecfe9c28765a652188f202a54"}
Jul 18 01:19:25.052235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:25.052199    2567 generic.go:356] "Generic (PLEG): container finished" podID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerID="523a5ef69163ea4328f2986a4016b2bc7b694b572903d84e8f6cdc29a9a78c4c" exitCode=6
Jul 18 01:19:25.052664 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:25.052311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" event={"ID":"48a58eaf-18a2-4499-9520-d39565c4e2b5","Type":"ContainerDied","Data":"523a5ef69163ea4328f2986a4016b2bc7b694b572903d84e8f6cdc29a9a78c4c"}
Jul 18 01:19:25.052664 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:25.052469    2567 scope.go:117] "RemoveContainer" containerID="523a5ef69163ea4328f2986a4016b2bc7b694b572903d84e8f6cdc29a9a78c4c"
Jul 18 01:19:26.059589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:26.059555    2567 generic.go:356] "Generic (PLEG): container finished" podID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerID="60c8170c4895a593269c8f3da9be2e9b8e0664ca831d5ea5a6c26fa2b22b4038" exitCode=6
Jul 18 01:19:26.060010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:26.059633    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" event={"ID":"48a58eaf-18a2-4499-9520-d39565c4e2b5","Type":"ContainerDied","Data":"60c8170c4895a593269c8f3da9be2e9b8e0664ca831d5ea5a6c26fa2b22b4038"}
Jul 18 01:19:26.060010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:26.059691    2567 scope.go:117] "RemoveContainer" containerID="523a5ef69163ea4328f2986a4016b2bc7b694b572903d84e8f6cdc29a9a78c4c"
Jul 18 01:19:26.060010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:26.059868    2567 scope.go:117] "RemoveContainer" containerID="60c8170c4895a593269c8f3da9be2e9b8e0664ca831d5ea5a6c26fa2b22b4038"
Jul 18 01:19:26.060154 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:26.060110    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-j466s_odh-ai-gateway-infra(48a58eaf-18a2-4499-9520-d39565c4e2b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5"
Jul 18 01:19:27.066547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:27.066518    2567 scope.go:117] "RemoveContainer" containerID="60c8170c4895a593269c8f3da9be2e9b8e0664ca831d5ea5a6c26fa2b22b4038"
Jul 18 01:19:27.066948 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:27.066718    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-j466s_odh-ai-gateway-infra(48a58eaf-18a2-4499-9520-d39565c4e2b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5"
Jul 18 01:19:35.890895 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.890860    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"]
Jul 18 01:19:35.891971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.891948    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:35.892086 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892076    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f41494-a2d0-46d6-8217-b1e7c104c2d2" containerName="revoke-keys"
Jul 18 01:19:35.892183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892174    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:35.892248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892241    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="091b0d0b-e399-4d61-9eab-abb86ea23212" containerName="revoke-keys"
Jul 18 01:19:35.892359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892348    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:35.892429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892421    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:35.892499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892491    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:35.892571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892563    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:35.892742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892733    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6811a73e-7b56-460b-89ee-7cef815d3f93" containerName="revoke-keys"
Jul 18 01:19:35.892818 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.892810    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0d192f-f3a0-47be-ae52-596b5a160183" containerName="revoke-keys"
Jul 18 01:19:35.912932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.912899    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"]
Jul 18 01:19:35.913180 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:35.913028    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:36.058105 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:36.058065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7daf789a-4d5d-48ac-a03a-a99b810cce73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l\" (UID: \"7daf789a-4d5d-48ac-a03a-a99b810cce73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:36.159314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:36.159185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7daf789a-4d5d-48ac-a03a-a99b810cce73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l\" (UID: \"7daf789a-4d5d-48ac-a03a-a99b810cce73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:36.159851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:36.159827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7daf789a-4d5d-48ac-a03a-a99b810cce73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l\" (UID: \"7daf789a-4d5d-48ac-a03a-a99b810cce73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:36.226233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:36.226195    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:36.360452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:36.360431    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"]
Jul 18 01:19:36.363637 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:19:36.363609    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7daf789a_4d5d_48ac_a03a_a99b810cce73.slice/crio-b6c59f3ade6ae4a5abfc7b4101c617fb51b137ce9faca7a94674163bffa6f6db WatchSource:0}: Error finding container b6c59f3ade6ae4a5abfc7b4101c617fb51b137ce9faca7a94674163bffa6f6db: Status 404 returned error can't find the container with id b6c59f3ade6ae4a5abfc7b4101c617fb51b137ce9faca7a94674163bffa6f6db
Jul 18 01:19:37.114742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:37.114705    2567 generic.go:356] "Generic (PLEG): container finished" podID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerID="7c653eb66951ef70cf6c24f0bf5c354b57eba1c85831ebd5737f3d3d516fff95" exitCode=7
Jul 18 01:19:37.115147 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:37.114795    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" event={"ID":"7daf789a-4d5d-48ac-a03a-a99b810cce73","Type":"ContainerDied","Data":"7c653eb66951ef70cf6c24f0bf5c354b57eba1c85831ebd5737f3d3d516fff95"}
Jul 18 01:19:37.115147 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:37.114865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" event={"ID":"7daf789a-4d5d-48ac-a03a-a99b810cce73","Type":"ContainerStarted","Data":"b6c59f3ade6ae4a5abfc7b4101c617fb51b137ce9faca7a94674163bffa6f6db"}
Jul 18 01:19:37.115147 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:37.114975    2567 scope.go:117] "RemoveContainer" containerID="7c653eb66951ef70cf6c24f0bf5c354b57eba1c85831ebd5737f3d3d516fff95"
Jul 18 01:19:37.913608 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:37.913572    2567 scope.go:117] "RemoveContainer" containerID="60c8170c4895a593269c8f3da9be2e9b8e0664ca831d5ea5a6c26fa2b22b4038"
Jul 18 01:19:38.124685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.124642    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" event={"ID":"48a58eaf-18a2-4499-9520-d39565c4e2b5","Type":"ContainerStarted","Data":"14e1315e2071f380bd3d5008c5422ec33815e713f7a488e94284ea37e4fe66aa"}
Jul 18 01:19:38.126729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.126657    2567 generic.go:356] "Generic (PLEG): container finished" podID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerID="b7fd63479e0918f98a42e1e8d28b32117cf68ff3cd753e1ef418704bf46aa93c" exitCode=7
Jul 18 01:19:38.126906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.126734    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" event={"ID":"7daf789a-4d5d-48ac-a03a-a99b810cce73","Type":"ContainerDied","Data":"b7fd63479e0918f98a42e1e8d28b32117cf68ff3cd753e1ef418704bf46aa93c"}
Jul 18 01:19:38.126906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.126828    2567 scope.go:117] "RemoveContainer" containerID="7c653eb66951ef70cf6c24f0bf5c354b57eba1c85831ebd5737f3d3d516fff95"
Jul 18 01:19:38.127036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.127019    2567 scope.go:117] "RemoveContainer" containerID="b7fd63479e0918f98a42e1e8d28b32117cf68ff3cd753e1ef418704bf46aa93c"
Jul 18 01:19:38.127307 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:38.127257    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l_odh-ai-gateway-infra(7daf789a-4d5d-48ac-a03a-a99b810cce73)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73"
Jul 18 01:19:38.140797 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.140741    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" podStartSLOduration=15.1407214 podStartE2EDuration="15.1407214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:19:38.138931745 +0000 UTC m=+2357.666436710" watchObservedRunningTime="2026-07-18 01:19:38.1407214 +0000 UTC m=+2357.668226364"
Jul 18 01:19:38.694556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.694472    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"]
Jul 18 01:19:38.697612 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.697594    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:19:38.704990 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.704961    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"]
Jul 18 01:19:38.880785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.880743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65fdda2c-4cc7-460d-a650-9966e7f72cc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj\" (UID: \"65fdda2c-4cc7-460d-a650-9966e7f72cc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:19:38.981627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.981525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65fdda2c-4cc7-460d-a650-9966e7f72cc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj\" (UID: \"65fdda2c-4cc7-460d-a650-9966e7f72cc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:19:38.982220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:38.982195    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65fdda2c-4cc7-460d-a650-9966e7f72cc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj\" (UID: \"65fdda2c-4cc7-460d-a650-9966e7f72cc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:19:39.008993 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.008947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:19:39.134136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.134106    2567 generic.go:356] "Generic (PLEG): container finished" podID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerID="14e1315e2071f380bd3d5008c5422ec33815e713f7a488e94284ea37e4fe66aa" exitCode=6
Jul 18 01:19:39.134633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.134177    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" event={"ID":"48a58eaf-18a2-4499-9520-d39565c4e2b5","Type":"ContainerDied","Data":"14e1315e2071f380bd3d5008c5422ec33815e713f7a488e94284ea37e4fe66aa"}
Jul 18 01:19:39.134633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.134250    2567 scope.go:117] "RemoveContainer" containerID="60c8170c4895a593269c8f3da9be2e9b8e0664ca831d5ea5a6c26fa2b22b4038"
Jul 18 01:19:39.134633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.134589    2567 scope.go:117] "RemoveContainer" containerID="14e1315e2071f380bd3d5008c5422ec33815e713f7a488e94284ea37e4fe66aa"
Jul 18 01:19:39.134865 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:39.134835    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-j466s_odh-ai-gateway-infra(48a58eaf-18a2-4499-9520-d39565c4e2b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5"
Jul 18 01:19:39.136116 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.136099    2567 scope.go:117] "RemoveContainer" containerID="b7fd63479e0918f98a42e1e8d28b32117cf68ff3cd753e1ef418704bf46aa93c"
Jul 18 01:19:39.136347 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:39.136328    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l_odh-ai-gateway-infra(7daf789a-4d5d-48ac-a03a-a99b810cce73)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73"
Jul 18 01:19:39.144965 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.144914    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"]
Jul 18 01:19:39.150426 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:19:39.150400    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65fdda2c_4cc7_460d_a650_9966e7f72cc5.slice/crio-6241f2fcd5ea8ea8727cc660e28e030917e7f98c8c42e2f78588e920c96a7819 WatchSource:0}: Error finding container 6241f2fcd5ea8ea8727cc660e28e030917e7f98c8c42e2f78588e920c96a7819: Status 404 returned error can't find the container with id 6241f2fcd5ea8ea8727cc660e28e030917e7f98c8c42e2f78588e920c96a7819
Jul 18 01:19:39.150925 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:39.150906    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"]
Jul 18 01:19:40.142241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.142146    2567 generic.go:356] "Generic (PLEG): container finished" podID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerID="24d339542a79c50953d79a561b9907d2253be14047d33f3887c909b20e559774" exitCode=6
Jul 18 01:19:40.142651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.142237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" event={"ID":"65fdda2c-4cc7-460d-a650-9966e7f72cc5","Type":"ContainerDied","Data":"24d339542a79c50953d79a561b9907d2253be14047d33f3887c909b20e559774"}
Jul 18 01:19:40.142651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.142323    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" event={"ID":"65fdda2c-4cc7-460d-a650-9966e7f72cc5","Type":"ContainerStarted","Data":"6241f2fcd5ea8ea8727cc660e28e030917e7f98c8c42e2f78588e920c96a7819"}
Jul 18 01:19:40.142651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.142447    2567 scope.go:117] "RemoveContainer" containerID="24d339542a79c50953d79a561b9907d2253be14047d33f3887c909b20e559774"
Jul 18 01:19:40.282450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.282423    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:40.393376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.393263    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a58eaf-18a2-4499-9520-d39565c4e2b5-maas-api-service-ca\") pod \"48a58eaf-18a2-4499-9520-d39565c4e2b5\" (UID: \"48a58eaf-18a2-4499-9520-d39565c4e2b5\") "
Jul 18 01:19:40.393674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.393652    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a58eaf-18a2-4499-9520-d39565c4e2b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48a58eaf-18a2-4499-9520-d39565c4e2b5" (UID: "48a58eaf-18a2-4499-9520-d39565c4e2b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:40.493901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.493865    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48a58eaf-18a2-4499-9520-d39565c4e2b5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:19:40.829773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.829724    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"]
Jul 18 01:19:40.830227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.830208    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:19:40.830346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.830230    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:19:40.830346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.830251    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:19:40.830346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.830259    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:19:40.830509 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.830375    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:19:40.830509 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.830393    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:19:40.833499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.833477    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:40.841879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.841402    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"]
Jul 18 01:19:40.998030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:40.997979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50562d74-02cd-4c55-8f2c-d4a4c31c030a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h\" (UID: \"50562d74-02cd-4c55-8f2c-d4a4c31c030a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:41.099731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.099617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50562d74-02cd-4c55-8f2c-d4a4c31c030a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h\" (UID: \"50562d74-02cd-4c55-8f2c-d4a4c31c030a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:41.100211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.100191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50562d74-02cd-4c55-8f2c-d4a4c31c030a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h\" (UID: \"50562d74-02cd-4c55-8f2c-d4a4c31c030a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:41.144882 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.144853    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:41.150512 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.150487    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"
Jul 18 01:19:41.150656 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.150532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s" event={"ID":"48a58eaf-18a2-4499-9520-d39565c4e2b5","Type":"ContainerDied","Data":"ed97aac742ed38571f6e94b6e0583399b32fbd2ecfe9c28765a652188f202a54"}
Jul 18 01:19:41.150656 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.150586    2567 scope.go:117] "RemoveContainer" containerID="14e1315e2071f380bd3d5008c5422ec33815e713f7a488e94284ea37e4fe66aa"
Jul 18 01:19:41.152756 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.152723    2567 generic.go:356] "Generic (PLEG): container finished" podID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerID="5be123a2603e83ab1f23e929cc1a4703902d214fd3c50a0db9677d82a4f75920" exitCode=6
Jul 18 01:19:41.152911 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.152766    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" event={"ID":"65fdda2c-4cc7-460d-a650-9966e7f72cc5","Type":"ContainerDied","Data":"5be123a2603e83ab1f23e929cc1a4703902d214fd3c50a0db9677d82a4f75920"}
Jul 18 01:19:41.153068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.153052    2567 scope.go:117] "RemoveContainer" containerID="5be123a2603e83ab1f23e929cc1a4703902d214fd3c50a0db9677d82a4f75920"
Jul 18 01:19:41.153437 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:41.153260    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj_odh-ai-gateway-infra(65fdda2c-4cc7-460d-a650-9966e7f72cc5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5"
Jul 18 01:19:41.176497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.176478    2567 scope.go:117] "RemoveContainer" containerID="24d339542a79c50953d79a561b9907d2253be14047d33f3887c909b20e559774"
Jul 18 01:19:41.179641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.179616    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"]
Jul 18 01:19:41.184027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.184001    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-j466s"]
Jul 18 01:19:41.288330 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:41.288303    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"]
Jul 18 01:19:41.290635 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:19:41.290598    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50562d74_02cd_4c55_8f2c_d4a4c31c030a.slice/crio-b333cda9eb1b525cfbcc26188e504759d51b170bedb0e56382357c020201a0a4 WatchSource:0}: Error finding container b333cda9eb1b525cfbcc26188e504759d51b170bedb0e56382357c020201a0a4: Status 404 returned error can't find the container with id b333cda9eb1b525cfbcc26188e504759d51b170bedb0e56382357c020201a0a4
Jul 18 01:19:42.160019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:42.159994    2567 scope.go:117] "RemoveContainer" containerID="5be123a2603e83ab1f23e929cc1a4703902d214fd3c50a0db9677d82a4f75920"
Jul 18 01:19:42.160514 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:42.160230    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj_odh-ai-gateway-infra(65fdda2c-4cc7-460d-a650-9966e7f72cc5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5"
Jul 18 01:19:42.162120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:42.162097    2567 generic.go:356] "Generic (PLEG): container finished" podID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerID="40f098a589e96d197c2ed13c74328eb80f6e51575d4458a1b5e736880ff96812" exitCode=6
Jul 18 01:19:42.162239 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:42.162156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" event={"ID":"50562d74-02cd-4c55-8f2c-d4a4c31c030a","Type":"ContainerDied","Data":"40f098a589e96d197c2ed13c74328eb80f6e51575d4458a1b5e736880ff96812"}
Jul 18 01:19:42.162239 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:42.162202    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" event={"ID":"50562d74-02cd-4c55-8f2c-d4a4c31c030a","Type":"ContainerStarted","Data":"b333cda9eb1b525cfbcc26188e504759d51b170bedb0e56382357c020201a0a4"}
Jul 18 01:19:42.162338 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:42.162315    2567 scope.go:117] "RemoveContainer" containerID="40f098a589e96d197c2ed13c74328eb80f6e51575d4458a1b5e736880ff96812"
Jul 18 01:19:42.918913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:42.918880    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" path="/var/lib/kubelet/pods/48a58eaf-18a2-4499-9520-d39565c4e2b5/volumes"
Jul 18 01:19:43.168910 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:43.168879    2567 generic.go:356] "Generic (PLEG): container finished" podID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerID="68c1bd111c77ed2bb9ab172194a933bd322f633e59e74841bc56d0590110781f" exitCode=6
Jul 18 01:19:43.169359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:43.168960    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" event={"ID":"50562d74-02cd-4c55-8f2c-d4a4c31c030a","Type":"ContainerDied","Data":"68c1bd111c77ed2bb9ab172194a933bd322f633e59e74841bc56d0590110781f"}
Jul 18 01:19:43.169359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:43.169016    2567 scope.go:117] "RemoveContainer" containerID="40f098a589e96d197c2ed13c74328eb80f6e51575d4458a1b5e736880ff96812"
Jul 18 01:19:43.169359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:43.169244    2567 scope.go:117] "RemoveContainer" containerID="68c1bd111c77ed2bb9ab172194a933bd322f633e59e74841bc56d0590110781f"
Jul 18 01:19:43.169534 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:43.169510    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h_odh-ai-gateway-infra(50562d74-02cd-4c55-8f2c-d4a4c31c030a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a"
Jul 18 01:19:44.175920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:44.175896    2567 scope.go:117] "RemoveContainer" containerID="68c1bd111c77ed2bb9ab172194a933bd322f633e59e74841bc56d0590110781f"
Jul 18 01:19:44.176376 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:44.176089    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h_odh-ai-gateway-infra(50562d74-02cd-4c55-8f2c-d4a4c31c030a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a"
Jul 18 01:19:52.913917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:52.913876    2567 scope.go:117] "RemoveContainer" containerID="b7fd63479e0918f98a42e1e8d28b32117cf68ff3cd753e1ef418704bf46aa93c"
Jul 18 01:19:54.230921 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:54.230890    2567 generic.go:356] "Generic (PLEG): container finished" podID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerID="c5d37b5fdcc36698aca8a2c3d182cf29466eba5d1df77cf4998d6833af441a7b" exitCode=7
Jul 18 01:19:54.231364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:54.230963    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" event={"ID":"7daf789a-4d5d-48ac-a03a-a99b810cce73","Type":"ContainerDied","Data":"c5d37b5fdcc36698aca8a2c3d182cf29466eba5d1df77cf4998d6833af441a7b"}
Jul 18 01:19:54.231364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:54.231026    2567 scope.go:117] "RemoveContainer" containerID="b7fd63479e0918f98a42e1e8d28b32117cf68ff3cd753e1ef418704bf46aa93c"
Jul 18 01:19:54.231364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:54.231320    2567 scope.go:117] "RemoveContainer" containerID="c5d37b5fdcc36698aca8a2c3d182cf29466eba5d1df77cf4998d6833af441a7b"
Jul 18 01:19:54.231566 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:54.231547    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l_odh-ai-gateway-infra(7daf789a-4d5d-48ac-a03a-a99b810cce73)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73"
Jul 18 01:19:55.255243 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:55.255211    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"]
Jul 18 01:19:55.400157 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:55.400127    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:55.431927 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:55.431894    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7daf789a-4d5d-48ac-a03a-a99b810cce73-maas-api-service-ca\") pod \"7daf789a-4d5d-48ac-a03a-a99b810cce73\" (UID: \"7daf789a-4d5d-48ac-a03a-a99b810cce73\") "
Jul 18 01:19:55.432287 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:55.432247    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7daf789a-4d5d-48ac-a03a-a99b810cce73-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7daf789a-4d5d-48ac-a03a-a99b810cce73" (UID: "7daf789a-4d5d-48ac-a03a-a99b810cce73"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:55.532499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:55.532412    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7daf789a-4d5d-48ac-a03a-a99b810cce73-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:19:55.913816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:55.913782    2567 scope.go:117] "RemoveContainer" containerID="68c1bd111c77ed2bb9ab172194a933bd322f633e59e74841bc56d0590110781f"
Jul 18 01:19:56.243568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.243538    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"
Jul 18 01:19:56.243568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.243558    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l" event={"ID":"7daf789a-4d5d-48ac-a03a-a99b810cce73","Type":"ContainerDied","Data":"b6c59f3ade6ae4a5abfc7b4101c617fb51b137ce9faca7a94674163bffa6f6db"}
Jul 18 01:19:56.243823 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.243625    2567 scope.go:117] "RemoveContainer" containerID="c5d37b5fdcc36698aca8a2c3d182cf29466eba5d1df77cf4998d6833af441a7b"
Jul 18 01:19:56.245854 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.245832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" event={"ID":"50562d74-02cd-4c55-8f2c-d4a4c31c030a","Type":"ContainerStarted","Data":"a54ba220ff756877a412f4aebac05541c94d6bae1e2ff249fa19cbd0be8528e9"}
Jul 18 01:19:56.264708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.264661    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" podStartSLOduration=16.264644627 podStartE2EDuration="16.264644627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:19:56.262714327 +0000 UTC m=+2375.790219287" watchObservedRunningTime="2026-07-18 01:19:56.264644627 +0000 UTC m=+2375.792149587"
Jul 18 01:19:56.278023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.277993    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"]
Jul 18 01:19:56.282403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.282380    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-n2x6l"]
Jul 18 01:19:56.913432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.913345    2567 scope.go:117] "RemoveContainer" containerID="5be123a2603e83ab1f23e929cc1a4703902d214fd3c50a0db9677d82a4f75920"
Jul 18 01:19:56.919474 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:56.919449    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" path="/var/lib/kubelet/pods/7daf789a-4d5d-48ac-a03a-a99b810cce73/volumes"
Jul 18 01:19:57.255040 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.255005    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" event={"ID":"65fdda2c-4cc7-460d-a650-9966e7f72cc5","Type":"ContainerStarted","Data":"5e29aa1c9044bf584fc5605f65466c43d55e771e82caaa5ce205bd310cf44a19"}
Jul 18 01:19:57.257857 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.257830    2567 generic.go:356] "Generic (PLEG): container finished" podID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerID="a54ba220ff756877a412f4aebac05541c94d6bae1e2ff249fa19cbd0be8528e9" exitCode=6
Jul 18 01:19:57.258028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.257854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" event={"ID":"50562d74-02cd-4c55-8f2c-d4a4c31c030a","Type":"ContainerDied","Data":"a54ba220ff756877a412f4aebac05541c94d6bae1e2ff249fa19cbd0be8528e9"}
Jul 18 01:19:57.258028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.257919    2567 scope.go:117] "RemoveContainer" containerID="68c1bd111c77ed2bb9ab172194a933bd322f633e59e74841bc56d0590110781f"
Jul 18 01:19:57.258201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.258184    2567 scope.go:117] "RemoveContainer" containerID="a54ba220ff756877a412f4aebac05541c94d6bae1e2ff249fa19cbd0be8528e9"
Jul 18 01:19:57.258451 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:57.258428    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h_odh-ai-gateway-infra(50562d74-02cd-4c55-8f2c-d4a4c31c030a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a"
Jul 18 01:19:57.271412 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.271387    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"]
Jul 18 01:19:57.274430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:57.274389    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" podStartSLOduration=19.274375794 podStartE2EDuration="19.274375794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:19:57.272595094 +0000 UTC m=+2376.800100055" watchObservedRunningTime="2026-07-18 01:19:57.274375794 +0000 UTC m=+2376.801880761"
Jul 18 01:19:58.265734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.265704    2567 generic.go:356] "Generic (PLEG): container finished" podID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerID="5e29aa1c9044bf584fc5605f65466c43d55e771e82caaa5ce205bd310cf44a19" exitCode=6
Jul 18 01:19:58.265931 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.265780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" event={"ID":"65fdda2c-4cc7-460d-a650-9966e7f72cc5","Type":"ContainerDied","Data":"5e29aa1c9044bf584fc5605f65466c43d55e771e82caaa5ce205bd310cf44a19"}
Jul 18 01:19:58.265931 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.265837    2567 scope.go:117] "RemoveContainer" containerID="5be123a2603e83ab1f23e929cc1a4703902d214fd3c50a0db9677d82a4f75920"
Jul 18 01:19:58.266165 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.266124    2567 scope.go:117] "RemoveContainer" containerID="5e29aa1c9044bf584fc5605f65466c43d55e771e82caaa5ce205bd310cf44a19"
Jul 18 01:19:58.266405 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:19:58.266381    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj_odh-ai-gateway-infra(65fdda2c-4cc7-460d-a650-9966e7f72cc5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5"
Jul 18 01:19:58.281143 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.281118    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"]
Jul 18 01:19:58.419815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.419791    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:58.459598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.459564    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50562d74-02cd-4c55-8f2c-d4a4c31c030a-maas-api-service-ca\") pod \"50562d74-02cd-4c55-8f2c-d4a4c31c030a\" (UID: \"50562d74-02cd-4c55-8f2c-d4a4c31c030a\") "
Jul 18 01:19:58.459911 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.459890    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50562d74-02cd-4c55-8f2c-d4a4c31c030a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50562d74-02cd-4c55-8f2c-d4a4c31c030a" (UID: "50562d74-02cd-4c55-8f2c-d4a4c31c030a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:58.560985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:58.560957    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50562d74-02cd-4c55-8f2c-d4a4c31c030a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:19:59.275638 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.275600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h" event={"ID":"50562d74-02cd-4c55-8f2c-d4a4c31c030a","Type":"ContainerDied","Data":"b333cda9eb1b525cfbcc26188e504759d51b170bedb0e56382357c020201a0a4"}
Jul 18 01:19:59.275781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.275665    2567 scope.go:117] "RemoveContainer" containerID="a54ba220ff756877a412f4aebac05541c94d6bae1e2ff249fa19cbd0be8528e9"
Jul 18 01:19:59.275781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.275609    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"
Jul 18 01:19:59.299611 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.299581    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"]
Jul 18 01:19:59.305431 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.305404    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-7d94h"]
Jul 18 01:19:59.421052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.421024    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:19:59.471037 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.471004    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65fdda2c-4cc7-460d-a650-9966e7f72cc5-maas-api-service-ca\") pod \"65fdda2c-4cc7-460d-a650-9966e7f72cc5\" (UID: \"65fdda2c-4cc7-460d-a650-9966e7f72cc5\") "
Jul 18 01:19:59.471466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.471441    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65fdda2c-4cc7-460d-a650-9966e7f72cc5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65fdda2c-4cc7-460d-a650-9966e7f72cc5" (UID: "65fdda2c-4cc7-460d-a650-9966e7f72cc5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:19:59.572633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:19:59.572595    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65fdda2c-4cc7-460d-a650-9966e7f72cc5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:20:00.281805 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.281769    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"
Jul 18 01:20:00.282016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.281814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj" event={"ID":"65fdda2c-4cc7-460d-a650-9966e7f72cc5","Type":"ContainerDied","Data":"6241f2fcd5ea8ea8727cc660e28e030917e7f98c8c42e2f78588e920c96a7819"}
Jul 18 01:20:00.282016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.281879    2567 scope.go:117] "RemoveContainer" containerID="5e29aa1c9044bf584fc5605f65466c43d55e771e82caaa5ce205bd310cf44a19"
Jul 18 01:20:00.307620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.307591    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"]
Jul 18 01:20:00.312233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.312209    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-w4cpj"]
Jul 18 01:20:00.918952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.918921    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" path="/var/lib/kubelet/pods/50562d74-02cd-4c55-8f2c-d4a4c31c030a/volumes"
Jul 18 01:20:00.919297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:00.919260    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" path="/var/lib/kubelet/pods/65fdda2c-4cc7-460d-a650-9966e7f72cc5/volumes"
Jul 18 01:20:13.328254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328213    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"]
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328594    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328605    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328624    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328629    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328638    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328644    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328652    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328657    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328665    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328670    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328676    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328681    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328686    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328692    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328697    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328702    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328765    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328772    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a58eaf-18a2-4499-9520-d39565c4e2b5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328778    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328783    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328790    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328798    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.328804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.328805    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:13.333042 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.333022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:13.335794 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.335770    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:20:13.335950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.335811    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:20:13.339580 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.339550    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"]
Jul 18 01:20:13.401610 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.401568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk\" (UID: \"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:13.502701 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.502661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk\" (UID: \"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:13.503357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.503338    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk\" (UID: \"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:13.646942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.646849    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:13.787711 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:13.787680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"]
Jul 18 01:20:13.789981 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:20:13.789951    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1cb87a_643e_45a1_8a08_ab1b1d4c2bf1.slice/crio-481d991291dbb56dbdda93c37c59fee9a6a1b1300a75203fa75ac8e7b3ebac17 WatchSource:0}: Error finding container 481d991291dbb56dbdda93c37c59fee9a6a1b1300a75203fa75ac8e7b3ebac17: Status 404 returned error can't find the container with id 481d991291dbb56dbdda93c37c59fee9a6a1b1300a75203fa75ac8e7b3ebac17
Jul 18 01:20:14.374223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:14.374196    2567 generic.go:356] "Generic (PLEG): container finished" podID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerID="ece5d082209fda8d93b1c57ea2b336f1acda2334b8df114d50bd27ae4c2d103e" exitCode=6
Jul 18 01:20:14.374571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:14.374294    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" event={"ID":"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1","Type":"ContainerDied","Data":"ece5d082209fda8d93b1c57ea2b336f1acda2334b8df114d50bd27ae4c2d103e"}
Jul 18 01:20:14.374571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:14.374349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" event={"ID":"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1","Type":"ContainerStarted","Data":"481d991291dbb56dbdda93c37c59fee9a6a1b1300a75203fa75ac8e7b3ebac17"}
Jul 18 01:20:14.374571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:14.374470    2567 scope.go:117] "RemoveContainer" containerID="ece5d082209fda8d93b1c57ea2b336f1acda2334b8df114d50bd27ae4c2d103e"
Jul 18 01:20:15.381161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:15.381134    2567 generic.go:356] "Generic (PLEG): container finished" podID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerID="9ca2c88adb4401168c92b8f4f0f982c07c7499e296e4cc10f66a4b84386c92fd" exitCode=6
Jul 18 01:20:15.381632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:15.381225    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" event={"ID":"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1","Type":"ContainerDied","Data":"9ca2c88adb4401168c92b8f4f0f982c07c7499e296e4cc10f66a4b84386c92fd"}
Jul 18 01:20:15.381632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:15.381318    2567 scope.go:117] "RemoveContainer" containerID="ece5d082209fda8d93b1c57ea2b336f1acda2334b8df114d50bd27ae4c2d103e"
Jul 18 01:20:15.381632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:15.381414    2567 scope.go:117] "RemoveContainer" containerID="9ca2c88adb4401168c92b8f4f0f982c07c7499e296e4cc10f66a4b84386c92fd"
Jul 18 01:20:15.381838 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:15.381647    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk_odh-ai-gateway-infra(2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1"
Jul 18 01:20:16.388281 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:16.388242    2567 scope.go:117] "RemoveContainer" containerID="9ca2c88adb4401168c92b8f4f0f982c07c7499e296e4cc10f66a4b84386c92fd"
Jul 18 01:20:16.388674 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:16.388458    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk_odh-ai-gateway-infra(2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1"
Jul 18 01:20:21.038069 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:21.038044    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:20:21.046592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:21.046567    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:20:28.913542 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:28.913512    2567 scope.go:117] "RemoveContainer" containerID="9ca2c88adb4401168c92b8f4f0f982c07c7499e296e4cc10f66a4b84386c92fd"
Jul 18 01:20:29.462437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:29.462400    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" event={"ID":"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1","Type":"ContainerStarted","Data":"3e52de63556ad017586bc3e74f5aca2aa3c7f8609d9b48c8b5a5bb639ccae7be"}
Jul 18 01:20:29.478231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:29.478185    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" podStartSLOduration=16.478169619 podStartE2EDuration="16.478169619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:20:29.476405596 +0000 UTC m=+2409.003910558" watchObservedRunningTime="2026-07-18 01:20:29.478169619 +0000 UTC m=+2409.005674579"
Jul 18 01:20:30.469006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.468977    2567 generic.go:356] "Generic (PLEG): container finished" podID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerID="3e52de63556ad017586bc3e74f5aca2aa3c7f8609d9b48c8b5a5bb639ccae7be" exitCode=6
Jul 18 01:20:30.469484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.469058    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" event={"ID":"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1","Type":"ContainerDied","Data":"3e52de63556ad017586bc3e74f5aca2aa3c7f8609d9b48c8b5a5bb639ccae7be"}
Jul 18 01:20:30.469484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.469130    2567 scope.go:117] "RemoveContainer" containerID="9ca2c88adb4401168c92b8f4f0f982c07c7499e296e4cc10f66a4b84386c92fd"
Jul 18 01:20:30.469484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.469444    2567 scope.go:117] "RemoveContainer" containerID="3e52de63556ad017586bc3e74f5aca2aa3c7f8609d9b48c8b5a5bb639ccae7be"
Jul 18 01:20:30.469691 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:30.469668    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk_odh-ai-gateway-infra(2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1"
Jul 18 01:20:30.487318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.487291    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"]
Jul 18 01:20:30.872834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.872803    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"]
Jul 18 01:20:30.873244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873230    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:30.873312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873246    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:30.873312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873254    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:30.873312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873260    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:30.873409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873342    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7daf789a-4d5d-48ac-a03a-a99b810cce73" containerName="revoke-keys"
Jul 18 01:20:30.873409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873354    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="65fdda2c-4cc7-460d-a650-9966e7f72cc5" containerName="revoke-keys"
Jul 18 01:20:30.873409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.873366    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="50562d74-02cd-4c55-8f2c-d4a4c31c030a" containerName="revoke-keys"
Jul 18 01:20:30.877516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.877498    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:30.883803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.883778    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"]
Jul 18 01:20:30.958552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:30.958514    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2d7216-0672-49de-afd7-2c3e93d79609-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7\" (UID: \"2f2d7216-0672-49de-afd7-2c3e93d79609\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:31.059715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.059683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2d7216-0672-49de-afd7-2c3e93d79609-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7\" (UID: \"2f2d7216-0672-49de-afd7-2c3e93d79609\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:31.060379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.060360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2d7216-0672-49de-afd7-2c3e93d79609-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7\" (UID: \"2f2d7216-0672-49de-afd7-2c3e93d79609\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:31.189226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.189149    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:31.328134 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.328100    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"]
Jul 18 01:20:31.331371 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:20:31.331334    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2d7216_0672_49de_afd7_2c3e93d79609.slice/crio-b9e3f0630aa13722bc7ec80ea8ef62d24960d21b977bdd8a4f6a8b8470edd869 WatchSource:0}: Error finding container b9e3f0630aa13722bc7ec80ea8ef62d24960d21b977bdd8a4f6a8b8470edd869: Status 404 returned error can't find the container with id b9e3f0630aa13722bc7ec80ea8ef62d24960d21b977bdd8a4f6a8b8470edd869
Jul 18 01:20:31.475700 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.475663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" event={"ID":"2f2d7216-0672-49de-afd7-2c3e93d79609","Type":"ContainerStarted","Data":"b9e3f0630aa13722bc7ec80ea8ef62d24960d21b977bdd8a4f6a8b8470edd869"}
Jul 18 01:20:31.614093 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.614069    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:31.664069 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.664044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1-maas-api-service-ca\") pod \"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1\" (UID: \"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1\") "
Jul 18 01:20:31.664411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.664388    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" (UID: "2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:20:31.765657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:31.765568    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:20:32.484124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.484040    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerID="12ce5b893933412397a8d84dc157821ec1aae1826ebfe9c0c9e33f5b08b222b1" exitCode=6
Jul 18 01:20:32.484593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.484133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" event={"ID":"2f2d7216-0672-49de-afd7-2c3e93d79609","Type":"ContainerDied","Data":"12ce5b893933412397a8d84dc157821ec1aae1826ebfe9c0c9e33f5b08b222b1"}
Jul 18 01:20:32.484593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.484378    2567 scope.go:117] "RemoveContainer" containerID="12ce5b893933412397a8d84dc157821ec1aae1826ebfe9c0c9e33f5b08b222b1"
Jul 18 01:20:32.485525 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.485508    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"
Jul 18 01:20:32.485625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.485555    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk" event={"ID":"2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1","Type":"ContainerDied","Data":"481d991291dbb56dbdda93c37c59fee9a6a1b1300a75203fa75ac8e7b3ebac17"}
Jul 18 01:20:32.485625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.485607    2567 scope.go:117] "RemoveContainer" containerID="3e52de63556ad017586bc3e74f5aca2aa3c7f8609d9b48c8b5a5bb639ccae7be"
Jul 18 01:20:32.517404 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.517378    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"]
Jul 18 01:20:32.523300 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.523263    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-nz8wk"]
Jul 18 01:20:32.919281 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:32.919245    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" path="/var/lib/kubelet/pods/2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1/volumes"
Jul 18 01:20:33.493362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:33.493248    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerID="cf92bd31b18e3f178343622afad2013abc8daacbd69eb9f256c2e7d1bcc8cab7" exitCode=6
Jul 18 01:20:33.493362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:33.493323    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" event={"ID":"2f2d7216-0672-49de-afd7-2c3e93d79609","Type":"ContainerDied","Data":"cf92bd31b18e3f178343622afad2013abc8daacbd69eb9f256c2e7d1bcc8cab7"}
Jul 18 01:20:33.493842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:33.493376    2567 scope.go:117] "RemoveContainer" containerID="12ce5b893933412397a8d84dc157821ec1aae1826ebfe9c0c9e33f5b08b222b1"
Jul 18 01:20:33.493842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:33.493559    2567 scope.go:117] "RemoveContainer" containerID="cf92bd31b18e3f178343622afad2013abc8daacbd69eb9f256c2e7d1bcc8cab7"
Jul 18 01:20:33.493842 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:33.493796    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7_odh-ai-gateway-infra(2f2d7216-0672-49de-afd7-2c3e93d79609)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609"
Jul 18 01:20:34.499594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:34.499570    2567 scope.go:117] "RemoveContainer" containerID="cf92bd31b18e3f178343622afad2013abc8daacbd69eb9f256c2e7d1bcc8cab7"
Jul 18 01:20:34.499974 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:34.499762    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7_odh-ai-gateway-infra(2f2d7216-0672-49de-afd7-2c3e93d79609)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609"
Jul 18 01:20:35.931845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.931811    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"]
Jul 18 01:20:35.932216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932205    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.932216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932216    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.932322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932233    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.932322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932238    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.932387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932327    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.932387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932339    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.932387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.932346    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:35.936557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.936538    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:35.941897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:35.941858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"]
Jul 18 01:20:36.005870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:36.005820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaa5fc2-65b5-4f63-8cd0-573656720a5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc\" (UID: \"daaa5fc2-65b5-4f63-8cd0-573656720a5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:36.106916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:36.106876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaa5fc2-65b5-4f63-8cd0-573656720a5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc\" (UID: \"daaa5fc2-65b5-4f63-8cd0-573656720a5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:36.107541 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:36.107520    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaa5fc2-65b5-4f63-8cd0-573656720a5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc\" (UID: \"daaa5fc2-65b5-4f63-8cd0-573656720a5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:36.248436 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:36.248350    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:36.385138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:36.385106    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"]
Jul 18 01:20:36.387666 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:20:36.387639    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaaa5fc2_65b5_4f63_8cd0_573656720a5f.slice/crio-2b9183d46221e9f5c57868f97d4e48ead495dae5e79a0f36f311a36bfce2a33c WatchSource:0}: Error finding container 2b9183d46221e9f5c57868f97d4e48ead495dae5e79a0f36f311a36bfce2a33c: Status 404 returned error can't find the container with id 2b9183d46221e9f5c57868f97d4e48ead495dae5e79a0f36f311a36bfce2a33c
Jul 18 01:20:36.510810 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:36.510717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" event={"ID":"daaa5fc2-65b5-4f63-8cd0-573656720a5f","Type":"ContainerStarted","Data":"2b9183d46221e9f5c57868f97d4e48ead495dae5e79a0f36f311a36bfce2a33c"}
Jul 18 01:20:37.517574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:37.517541    2567 generic.go:356] "Generic (PLEG): container finished" podID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerID="6574788458cefe1b7e6e3137f7569e33c6fca379fdfd8fc12e1d9e48a7ee82e9" exitCode=7
Jul 18 01:20:37.517970 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:37.517639    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" event={"ID":"daaa5fc2-65b5-4f63-8cd0-573656720a5f","Type":"ContainerDied","Data":"6574788458cefe1b7e6e3137f7569e33c6fca379fdfd8fc12e1d9e48a7ee82e9"}
Jul 18 01:20:37.517970 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:37.517827    2567 scope.go:117] "RemoveContainer" containerID="6574788458cefe1b7e6e3137f7569e33c6fca379fdfd8fc12e1d9e48a7ee82e9"
Jul 18 01:20:38.525558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.525463    2567 generic.go:356] "Generic (PLEG): container finished" podID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerID="5e3d1b860b5499e3a8cf2d6ab0bf638ee8dbc44eda46baec832d3bae5916210b" exitCode=7
Jul 18 01:20:38.525971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.525551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" event={"ID":"daaa5fc2-65b5-4f63-8cd0-573656720a5f","Type":"ContainerDied","Data":"5e3d1b860b5499e3a8cf2d6ab0bf638ee8dbc44eda46baec832d3bae5916210b"}
Jul 18 01:20:38.525971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.525637    2567 scope.go:117] "RemoveContainer" containerID="6574788458cefe1b7e6e3137f7569e33c6fca379fdfd8fc12e1d9e48a7ee82e9"
Jul 18 01:20:38.525971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.525757    2567 scope.go:117] "RemoveContainer" containerID="5e3d1b860b5499e3a8cf2d6ab0bf638ee8dbc44eda46baec832d3bae5916210b"
Jul 18 01:20:38.526088 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:38.525990    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc_odh-ai-gateway-infra(daaa5fc2-65b5-4f63-8cd0-573656720a5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f"
Jul 18 01:20:38.740534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.740497    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"]
Jul 18 01:20:38.740981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.740959    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:38.741081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.740984    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1cb87a-643e-45a1-8a08-ab1b1d4c2bf1" containerName="revoke-keys"
Jul 18 01:20:38.745367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.745343    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:38.750094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.750059    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"]
Jul 18 01:20:38.829972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.829929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b4c66-7497-4e57-a2d4-08790bf48855-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk\" (UID: \"9e0b4c66-7497-4e57-a2d4-08790bf48855\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:38.931075 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.931037    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b4c66-7497-4e57-a2d4-08790bf48855-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk\" (UID: \"9e0b4c66-7497-4e57-a2d4-08790bf48855\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:38.931651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:38.931632    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b4c66-7497-4e57-a2d4-08790bf48855-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk\" (UID: \"9e0b4c66-7497-4e57-a2d4-08790bf48855\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:39.058325 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:39.058289    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:39.202952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:39.202928    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"]
Jul 18 01:20:39.204846 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:20:39.204821    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0b4c66_7497_4e57_a2d4_08790bf48855.slice/crio-1463c690eff7c3a67be265b6da13977b11d6579ac5084e9218d67c93b710a35a WatchSource:0}: Error finding container 1463c690eff7c3a67be265b6da13977b11d6579ac5084e9218d67c93b710a35a: Status 404 returned error can't find the container with id 1463c690eff7c3a67be265b6da13977b11d6579ac5084e9218d67c93b710a35a
Jul 18 01:20:39.532045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:39.532008    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerStarted","Data":"c5beea342f0879c3691baceb5da40cc2e084aaab1dedebcf8cccb0a0d6010329"}
Jul 18 01:20:39.532526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:39.532079    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerStarted","Data":"1463c690eff7c3a67be265b6da13977b11d6579ac5084e9218d67c93b710a35a"}
Jul 18 01:20:39.533821 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:39.533797    2567 scope.go:117] "RemoveContainer" containerID="5e3d1b860b5499e3a8cf2d6ab0bf638ee8dbc44eda46baec832d3bae5916210b"
Jul 18 01:20:39.534008 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:39.533989    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc_odh-ai-gateway-infra(daaa5fc2-65b5-4f63-8cd0-573656720a5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f"
Jul 18 01:20:39.560376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:39.560334    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" podStartSLOduration=1.560319624 podStartE2EDuration="1.560319624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:20:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:20:39.545244231 +0000 UTC m=+2419.072749188" watchObservedRunningTime="2026-07-18 01:20:39.560319624 +0000 UTC m=+2419.087824588"
Jul 18 01:20:40.541192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:40.541159    2567 generic.go:356] "Generic (PLEG): container finished" podID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerID="c5beea342f0879c3691baceb5da40cc2e084aaab1dedebcf8cccb0a0d6010329" exitCode=6
Jul 18 01:20:40.541654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:40.541247    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerDied","Data":"c5beea342f0879c3691baceb5da40cc2e084aaab1dedebcf8cccb0a0d6010329"}
Jul 18 01:20:40.541654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:40.541433    2567 scope.go:117] "RemoveContainer" containerID="c5beea342f0879c3691baceb5da40cc2e084aaab1dedebcf8cccb0a0d6010329"
Jul 18 01:20:41.548919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:41.548830    2567 generic.go:356] "Generic (PLEG): container finished" podID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerID="46c595b1582e8c760e7d9d41e192058efb843082c8bdff0d8075707718c41eee" exitCode=6
Jul 18 01:20:41.549328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:41.548918    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerDied","Data":"46c595b1582e8c760e7d9d41e192058efb843082c8bdff0d8075707718c41eee"}
Jul 18 01:20:41.549328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:41.548968    2567 scope.go:117] "RemoveContainer" containerID="c5beea342f0879c3691baceb5da40cc2e084aaab1dedebcf8cccb0a0d6010329"
Jul 18 01:20:41.549328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:41.549169    2567 scope.go:117] "RemoveContainer" containerID="46c595b1582e8c760e7d9d41e192058efb843082c8bdff0d8075707718c41eee"
Jul 18 01:20:41.549465 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:41.549452    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk_odh-ai-gateway-infra(9e0b4c66-7497-4e57-a2d4-08790bf48855)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855"
Jul 18 01:20:42.556408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:42.556380    2567 scope.go:117] "RemoveContainer" containerID="46c595b1582e8c760e7d9d41e192058efb843082c8bdff0d8075707718c41eee"
Jul 18 01:20:42.556797 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:42.556590    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk_odh-ai-gateway-infra(9e0b4c66-7497-4e57-a2d4-08790bf48855)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855"
Jul 18 01:20:45.913722 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:45.913688    2567 scope.go:117] "RemoveContainer" containerID="cf92bd31b18e3f178343622afad2013abc8daacbd69eb9f256c2e7d1bcc8cab7"
Jul 18 01:20:46.577885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:46.577849    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerID="6e981e7ff2332eb08362c01e90ac9c939d4608a0881ef649860df559453cd42a" exitCode=6
Jul 18 01:20:46.578089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:46.577890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" event={"ID":"2f2d7216-0672-49de-afd7-2c3e93d79609","Type":"ContainerDied","Data":"6e981e7ff2332eb08362c01e90ac9c939d4608a0881ef649860df559453cd42a"}
Jul 18 01:20:46.578089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:46.577952    2567 scope.go:117] "RemoveContainer" containerID="cf92bd31b18e3f178343622afad2013abc8daacbd69eb9f256c2e7d1bcc8cab7"
Jul 18 01:20:46.578247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:46.578227    2567 scope.go:117] "RemoveContainer" containerID="6e981e7ff2332eb08362c01e90ac9c939d4608a0881ef649860df559453cd42a"
Jul 18 01:20:46.578479 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:46.578458    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7_odh-ai-gateway-infra(2f2d7216-0672-49de-afd7-2c3e93d79609)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609"
Jul 18 01:20:47.602622 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:47.602587    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"]
Jul 18 01:20:47.744773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:47.744747    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:47.818001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:47.817962    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2d7216-0672-49de-afd7-2c3e93d79609-maas-api-service-ca\") pod \"2f2d7216-0672-49de-afd7-2c3e93d79609\" (UID: \"2f2d7216-0672-49de-afd7-2c3e93d79609\") "
Jul 18 01:20:47.818367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:47.818344    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2d7216-0672-49de-afd7-2c3e93d79609-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f2d7216-0672-49de-afd7-2c3e93d79609" (UID: "2f2d7216-0672-49de-afd7-2c3e93d79609"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:20:47.919312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:47.919195    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2d7216-0672-49de-afd7-2c3e93d79609-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:20:48.591465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:48.591435    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"
Jul 18 01:20:48.591671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:48.591436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7" event={"ID":"2f2d7216-0672-49de-afd7-2c3e93d79609","Type":"ContainerDied","Data":"b9e3f0630aa13722bc7ec80ea8ef62d24960d21b977bdd8a4f6a8b8470edd869"}
Jul 18 01:20:48.591671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:48.591571    2567 scope.go:117] "RemoveContainer" containerID="6e981e7ff2332eb08362c01e90ac9c939d4608a0881ef649860df559453cd42a"
Jul 18 01:20:48.614548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:48.614519    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"]
Jul 18 01:20:48.620796 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:48.620770    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-w66j7"]
Jul 18 01:20:48.918899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:48.918816    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" path="/var/lib/kubelet/pods/2f2d7216-0672-49de-afd7-2c3e93d79609/volumes"
Jul 18 01:20:53.913477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:53.913444    2567 scope.go:117] "RemoveContainer" containerID="46c595b1582e8c760e7d9d41e192058efb843082c8bdff0d8075707718c41eee"
Jul 18 01:20:53.913880 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:53.913579    2567 scope.go:117] "RemoveContainer" containerID="5e3d1b860b5499e3a8cf2d6ab0bf638ee8dbc44eda46baec832d3bae5916210b"
Jul 18 01:20:54.625977 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:54.625945    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerStarted","Data":"0246fa1af69149137a72b762f7c975d552bb561cf49944714fe167a3a101a6fc"}
Jul 18 01:20:54.627711 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:54.627687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" event={"ID":"daaa5fc2-65b5-4f63-8cd0-573656720a5f","Type":"ContainerStarted","Data":"8033204ffd04b72bc2147ebc1a58acc8788e0af9a058bd107c1a34ee586c338e"}
Jul 18 01:20:54.653222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:54.653175    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" podStartSLOduration=19.653155963 podStartE2EDuration="19.653155963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:20:54.652717587 +0000 UTC m=+2434.180222565" watchObservedRunningTime="2026-07-18 01:20:54.653155963 +0000 UTC m=+2434.180660923"
Jul 18 01:20:55.634615 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.634582    2567 generic.go:356] "Generic (PLEG): container finished" podID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerID="0246fa1af69149137a72b762f7c975d552bb561cf49944714fe167a3a101a6fc" exitCode=6
Jul 18 01:20:55.635088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.634654    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerDied","Data":"0246fa1af69149137a72b762f7c975d552bb561cf49944714fe167a3a101a6fc"}
Jul 18 01:20:55.635088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.634712    2567 scope.go:117] "RemoveContainer" containerID="46c595b1582e8c760e7d9d41e192058efb843082c8bdff0d8075707718c41eee"
Jul 18 01:20:55.635088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.634968    2567 scope.go:117] "RemoveContainer" containerID="0246fa1af69149137a72b762f7c975d552bb561cf49944714fe167a3a101a6fc"
Jul 18 01:20:55.635294 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:55.635205    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk_odh-ai-gateway-infra(9e0b4c66-7497-4e57-a2d4-08790bf48855)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855"
Jul 18 01:20:55.636830 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.636812    2567 generic.go:356] "Generic (PLEG): container finished" podID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerID="8033204ffd04b72bc2147ebc1a58acc8788e0af9a058bd107c1a34ee586c338e" exitCode=7
Jul 18 01:20:55.636924 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.636856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" event={"ID":"daaa5fc2-65b5-4f63-8cd0-573656720a5f","Type":"ContainerDied","Data":"8033204ffd04b72bc2147ebc1a58acc8788e0af9a058bd107c1a34ee586c338e"}
Jul 18 01:20:55.637073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.637062    2567 scope.go:117] "RemoveContainer" containerID="8033204ffd04b72bc2147ebc1a58acc8788e0af9a058bd107c1a34ee586c338e"
Jul 18 01:20:55.637242 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:20:55.637222    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc_odh-ai-gateway-infra(daaa5fc2-65b5-4f63-8cd0-573656720a5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f"
Jul 18 01:20:55.646235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.646216    2567 scope.go:117] "RemoveContainer" containerID="5e3d1b860b5499e3a8cf2d6ab0bf638ee8dbc44eda46baec832d3bae5916210b"
Jul 18 01:20:55.650512 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.650491    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"]
Jul 18 01:20:55.664984 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:55.664957    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"]
Jul 18 01:20:56.788605 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:56.788580    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:56.806650 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:56.806621    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:56.908567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:56.908466    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaa5fc2-65b5-4f63-8cd0-573656720a5f-maas-api-service-ca\") pod \"daaa5fc2-65b5-4f63-8cd0-573656720a5f\" (UID: \"daaa5fc2-65b5-4f63-8cd0-573656720a5f\") "
Jul 18 01:20:56.908731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:56.908589    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b4c66-7497-4e57-a2d4-08790bf48855-maas-api-service-ca\") pod \"9e0b4c66-7497-4e57-a2d4-08790bf48855\" (UID: \"9e0b4c66-7497-4e57-a2d4-08790bf48855\") "
Jul 18 01:20:56.908923 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:56.908894    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daaa5fc2-65b5-4f63-8cd0-573656720a5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "daaa5fc2-65b5-4f63-8cd0-573656720a5f" (UID: "daaa5fc2-65b5-4f63-8cd0-573656720a5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:20:56.908963 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:56.908923    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e0b4c66-7497-4e57-a2d4-08790bf48855-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e0b4c66-7497-4e57-a2d4-08790bf48855" (UID: "9e0b4c66-7497-4e57-a2d4-08790bf48855"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:20:57.009645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.009609    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daaa5fc2-65b5-4f63-8cd0-573656720a5f-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:20:57.009645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.009639    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0b4c66-7497-4e57-a2d4-08790bf48855-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:20:57.651700 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.651653    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk" event={"ID":"9e0b4c66-7497-4e57-a2d4-08790bf48855","Type":"ContainerDied","Data":"1463c690eff7c3a67be265b6da13977b11d6579ac5084e9218d67c93b710a35a"}
Jul 18 01:20:57.651700 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.651682    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"
Jul 18 01:20:57.651943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.651737    2567 scope.go:117] "RemoveContainer" containerID="0246fa1af69149137a72b762f7c975d552bb561cf49944714fe167a3a101a6fc"
Jul 18 01:20:57.653084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.652961    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc" event={"ID":"daaa5fc2-65b5-4f63-8cd0-573656720a5f","Type":"ContainerDied","Data":"2b9183d46221e9f5c57868f97d4e48ead495dae5e79a0f36f311a36bfce2a33c"}
Jul 18 01:20:57.653084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.652997    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"
Jul 18 01:20:57.661090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.661069    2567 scope.go:117] "RemoveContainer" containerID="8033204ffd04b72bc2147ebc1a58acc8788e0af9a058bd107c1a34ee586c338e"
Jul 18 01:20:57.670654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.670626    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"]
Jul 18 01:20:57.672559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.672537    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-zdcbc"]
Jul 18 01:20:57.684309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.684263    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"]
Jul 18 01:20:57.693321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:57.693290    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cftxk"]
Jul 18 01:20:58.919777 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:58.919746    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" path="/var/lib/kubelet/pods/9e0b4c66-7497-4e57-a2d4-08790bf48855/volumes"
Jul 18 01:20:58.920169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:20:58.920085    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" path="/var/lib/kubelet/pods/daaa5fc2-65b5-4f63-8cd0-573656720a5f/volumes"
Jul 18 01:21:00.689833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:00.689799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:03.371204 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371171    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"]
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371559    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371571    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371582    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371589    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371599    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371605    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371613    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371619    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371631    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371636    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371644    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371649    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371653    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371659    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371666    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371671    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371729    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371736    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371741    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371746    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371753    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371761    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.371866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.371766    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="daaa5fc2-65b5-4f63-8cd0-573656720a5f" containerName="revoke-keys"
Jul 18 01:21:03.376261 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.376241    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:03.378844 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.378820    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:21:03.379676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.379650    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:21:03.382388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.382362    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"]
Jul 18 01:21:03.466255 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.466218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-dpntr\" (UID: \"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:03.567366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.567312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-dpntr\" (UID: \"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:03.567956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.567937    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-dpntr\" (UID: \"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:03.688881 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.688788    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:03.829868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:03.829679    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"]
Jul 18 01:21:03.832799 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:21:03.832771    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2cea0ac_8ee6_42ba_ac0c_e04e9f1eb011.slice/crio-f1572eb86448298658ae2d3980d07495baa11069cd5119a811f41308945d6a06 WatchSource:0}: Error finding container f1572eb86448298658ae2d3980d07495baa11069cd5119a811f41308945d6a06: Status 404 returned error can't find the container with id f1572eb86448298658ae2d3980d07495baa11069cd5119a811f41308945d6a06
Jul 18 01:21:04.691507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:04.691471    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerID="8fdcfb311bc7df9189a375201564de3cc79b38a4d88b4bdb573a16d5911c3af9" exitCode=6
Jul 18 01:21:04.691901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:04.691554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" event={"ID":"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011","Type":"ContainerDied","Data":"8fdcfb311bc7df9189a375201564de3cc79b38a4d88b4bdb573a16d5911c3af9"}
Jul 18 01:21:04.691901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:04.691596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" event={"ID":"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011","Type":"ContainerStarted","Data":"f1572eb86448298658ae2d3980d07495baa11069cd5119a811f41308945d6a06"}
Jul 18 01:21:04.691901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:04.691747    2567 scope.go:117] "RemoveContainer" containerID="8fdcfb311bc7df9189a375201564de3cc79b38a4d88b4bdb573a16d5911c3af9"
Jul 18 01:21:05.698199 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:05.698114    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerID="3f51a328fd024bffcddcd228b84cfd45567ed2852d7a2d31845784d560574748" exitCode=6
Jul 18 01:21:05.698620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:05.698202    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" event={"ID":"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011","Type":"ContainerDied","Data":"3f51a328fd024bffcddcd228b84cfd45567ed2852d7a2d31845784d560574748"}
Jul 18 01:21:05.698620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:05.698257    2567 scope.go:117] "RemoveContainer" containerID="8fdcfb311bc7df9189a375201564de3cc79b38a4d88b4bdb573a16d5911c3af9"
Jul 18 01:21:05.698620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:05.698399    2567 scope.go:117] "RemoveContainer" containerID="3f51a328fd024bffcddcd228b84cfd45567ed2852d7a2d31845784d560574748"
Jul 18 01:21:05.698620 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:05.698604    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-dpntr_odh-ai-gateway-infra(a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011"
Jul 18 01:21:06.706071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:06.706039    2567 scope.go:117] "RemoveContainer" containerID="3f51a328fd024bffcddcd228b84cfd45567ed2852d7a2d31845784d560574748"
Jul 18 01:21:06.706526 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:06.706248    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-dpntr_odh-ai-gateway-infra(a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011"
Jul 18 01:21:07.038344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:07.038201    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:07.142164 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:07.142129    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:20.915686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.915649    2567 scope.go:117] "RemoveContainer" containerID="3f51a328fd024bffcddcd228b84cfd45567ed2852d7a2d31845784d560574748"
Jul 18 01:21:20.920819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.920792    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"]
Jul 18 01:21:20.921450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.921424    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:20.921559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.921453    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:20.921559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.921554    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0b4c66-7497-4e57-a2d4-08790bf48855" containerName="revoke-keys"
Jul 18 01:21:20.921638 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.921571    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2d7216-0672-49de-afd7-2c3e93d79609" containerName="revoke-keys"
Jul 18 01:21:20.926356 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.926332    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"]
Jul 18 01:21:20.926475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:20.926449    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:21.025874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.025827    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988b847a-783b-43f7-bcd9-d921ce3e19fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh\" (UID: \"988b847a-783b-43f7-bcd9-d921ce3e19fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:21.126564 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.126526    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988b847a-783b-43f7-bcd9-d921ce3e19fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh\" (UID: \"988b847a-783b-43f7-bcd9-d921ce3e19fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:21.127189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.127164    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988b847a-783b-43f7-bcd9-d921ce3e19fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh\" (UID: \"988b847a-783b-43f7-bcd9-d921ce3e19fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:21.275455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.275417    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:21.420156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.420133    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"]
Jul 18 01:21:21.422401 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:21:21.422340    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988b847a_783b_43f7_bcd9_d921ce3e19fa.slice/crio-ac9710775892606c070a59394bee4cf043aead9fd3eef4636e353a12fb06bc87 WatchSource:0}: Error finding container ac9710775892606c070a59394bee4cf043aead9fd3eef4636e353a12fb06bc87: Status 404 returned error can't find the container with id ac9710775892606c070a59394bee4cf043aead9fd3eef4636e353a12fb06bc87
Jul 18 01:21:21.784620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.784584    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" event={"ID":"988b847a-783b-43f7-bcd9-d921ce3e19fa","Type":"ContainerStarted","Data":"a736dcdf5adbccdbd36b61ef10a560de8fd5634da200c9686a1e8fa1ec73c313"}
Jul 18 01:21:21.784821 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.784648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" event={"ID":"988b847a-783b-43f7-bcd9-d921ce3e19fa","Type":"ContainerStarted","Data":"ac9710775892606c070a59394bee4cf043aead9fd3eef4636e353a12fb06bc87"}
Jul 18 01:21:21.786479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.786454    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerID="378a1f92a845aeee341c66317232d23b1e5ec28e26797c44b6f32dee3b59df0e" exitCode=6
Jul 18 01:21:21.786598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.786491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" event={"ID":"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011","Type":"ContainerDied","Data":"378a1f92a845aeee341c66317232d23b1e5ec28e26797c44b6f32dee3b59df0e"}
Jul 18 01:21:21.786598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.786533    2567 scope.go:117] "RemoveContainer" containerID="3f51a328fd024bffcddcd228b84cfd45567ed2852d7a2d31845784d560574748"
Jul 18 01:21:21.786791 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.786775    2567 scope.go:117] "RemoveContainer" containerID="378a1f92a845aeee341c66317232d23b1e5ec28e26797c44b6f32dee3b59df0e"
Jul 18 01:21:21.787042 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:21.787016    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-dpntr_odh-ai-gateway-infra(a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011"
Jul 18 01:21:21.799838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:21.799779    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" podStartSLOduration=1.799758589 podStartE2EDuration="1.799758589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:21:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:21:21.799615413 +0000 UTC m=+2461.327120374" watchObservedRunningTime="2026-07-18 01:21:21.799758589 +0000 UTC m=+2461.327263552"
Jul 18 01:21:22.793111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:22.793074    2567 generic.go:356] "Generic (PLEG): container finished" podID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerID="a736dcdf5adbccdbd36b61ef10a560de8fd5634da200c9686a1e8fa1ec73c313" exitCode=6
Jul 18 01:21:22.793590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:22.793114    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" event={"ID":"988b847a-783b-43f7-bcd9-d921ce3e19fa","Type":"ContainerDied","Data":"a736dcdf5adbccdbd36b61ef10a560de8fd5634da200c9686a1e8fa1ec73c313"}
Jul 18 01:21:22.793590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:22.793397    2567 scope.go:117] "RemoveContainer" containerID="a736dcdf5adbccdbd36b61ef10a560de8fd5634da200c9686a1e8fa1ec73c313"
Jul 18 01:21:22.831041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:22.831014    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"]
Jul 18 01:21:22.979489 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:22.979460    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:23.043296 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.043173    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011-maas-api-service-ca\") pod \"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011\" (UID: \"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011\") "
Jul 18 01:21:23.043562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.043538    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" (UID: "a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:21:23.144092 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.144057    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:23.802054 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.801966    2567 generic.go:356] "Generic (PLEG): container finished" podID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerID="8883b403cee2014c34a4421e1a2a152e5565aa3d089cb3de1d2d88e727b4471f" exitCode=6
Jul 18 01:21:23.802054 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.802022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" event={"ID":"988b847a-783b-43f7-bcd9-d921ce3e19fa","Type":"ContainerDied","Data":"8883b403cee2014c34a4421e1a2a152e5565aa3d089cb3de1d2d88e727b4471f"}
Jul 18 01:21:23.802620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.802083    2567 scope.go:117] "RemoveContainer" containerID="a736dcdf5adbccdbd36b61ef10a560de8fd5634da200c9686a1e8fa1ec73c313"
Jul 18 01:21:23.802620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.802317    2567 scope.go:117] "RemoveContainer" containerID="8883b403cee2014c34a4421e1a2a152e5565aa3d089cb3de1d2d88e727b4471f"
Jul 18 01:21:23.802620 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:23.802553    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh_odh-ai-gateway-infra(988b847a-783b-43f7-bcd9-d921ce3e19fa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa"
Jul 18 01:21:23.803548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.803529    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"
Jul 18 01:21:23.803634 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.803553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr" event={"ID":"a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011","Type":"ContainerDied","Data":"f1572eb86448298658ae2d3980d07495baa11069cd5119a811f41308945d6a06"}
Jul 18 01:21:23.814105 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.814087    2567 scope.go:117] "RemoveContainer" containerID="378a1f92a845aeee341c66317232d23b1e5ec28e26797c44b6f32dee3b59df0e"
Jul 18 01:21:23.829067 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.829040    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"]
Jul 18 01:21:23.832790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:23.832768    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-dpntr"]
Jul 18 01:21:24.811519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:24.811495    2567 scope.go:117] "RemoveContainer" containerID="8883b403cee2014c34a4421e1a2a152e5565aa3d089cb3de1d2d88e727b4471f"
Jul 18 01:21:24.812022 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:24.811743    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh_odh-ai-gateway-infra(988b847a-783b-43f7-bcd9-d921ce3e19fa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa"
Jul 18 01:21:24.919062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:24.919027    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" path="/var/lib/kubelet/pods/a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011/volumes"
Jul 18 01:21:26.001890 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:26.001831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:26.105674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:26.105630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:28.781448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781411    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"]
Jul 18 01:21:28.781835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781825    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:28.781875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781838    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:28.781875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781848    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:28.781875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781854    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:28.781997 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781941    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:28.781997 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.781951    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:28.785105 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.785086    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:28.791415 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.791388    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"]
Jul 18 01:21:28.893183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.893141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40c4e4e-940a-482b-91d9-0ef8c39953a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx\" (UID: \"e40c4e4e-940a-482b-91d9-0ef8c39953a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:28.993869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.993832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40c4e4e-940a-482b-91d9-0ef8c39953a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx\" (UID: \"e40c4e4e-940a-482b-91d9-0ef8c39953a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:28.994516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:28.994497    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40c4e4e-940a-482b-91d9-0ef8c39953a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx\" (UID: \"e40c4e4e-940a-482b-91d9-0ef8c39953a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:29.097142 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:29.097106    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:29.236704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:29.236678    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"]
Jul 18 01:21:29.238823 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:21:29.238797    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40c4e4e_940a_482b_91d9_0ef8c39953a4.slice/crio-3e2c620da995848d03c1fcef682d36cce470572094d23bff1c0712edba1dabfc WatchSource:0}: Error finding container 3e2c620da995848d03c1fcef682d36cce470572094d23bff1c0712edba1dabfc: Status 404 returned error can't find the container with id 3e2c620da995848d03c1fcef682d36cce470572094d23bff1c0712edba1dabfc
Jul 18 01:21:29.839731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:29.839695    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" event={"ID":"e40c4e4e-940a-482b-91d9-0ef8c39953a4","Type":"ContainerStarted","Data":"fcd423e616a704638a6820de921b67a2acf2fe931e7f38657ce801e38953ca94"}
Jul 18 01:21:29.840197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:29.839744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" event={"ID":"e40c4e4e-940a-482b-91d9-0ef8c39953a4","Type":"ContainerStarted","Data":"3e2c620da995848d03c1fcef682d36cce470572094d23bff1c0712edba1dabfc"}
Jul 18 01:21:29.854954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:29.854905    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" podStartSLOduration=1.85488422 podStartE2EDuration="1.85488422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:21:29.852719741 +0000 UTC m=+2469.380224701" watchObservedRunningTime="2026-07-18 01:21:29.85488422 +0000 UTC m=+2469.382389179"
Jul 18 01:21:30.845820 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:30.845782    2567 generic.go:356] "Generic (PLEG): container finished" podID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerID="fcd423e616a704638a6820de921b67a2acf2fe931e7f38657ce801e38953ca94" exitCode=6
Jul 18 01:21:30.846285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:30.845868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" event={"ID":"e40c4e4e-940a-482b-91d9-0ef8c39953a4","Type":"ContainerDied","Data":"fcd423e616a704638a6820de921b67a2acf2fe931e7f38657ce801e38953ca94"}
Jul 18 01:21:30.846285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:30.846012    2567 scope.go:117] "RemoveContainer" containerID="fcd423e616a704638a6820de921b67a2acf2fe931e7f38657ce801e38953ca94"
Jul 18 01:21:31.853235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:31.853191    2567 generic.go:356] "Generic (PLEG): container finished" podID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerID="13e4c2c2618c7823c02c9ff87b19e2ee03e63b1e50837329a88fffc8d299659d" exitCode=6
Jul 18 01:21:31.853746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:31.853289    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" event={"ID":"e40c4e4e-940a-482b-91d9-0ef8c39953a4","Type":"ContainerDied","Data":"13e4c2c2618c7823c02c9ff87b19e2ee03e63b1e50837329a88fffc8d299659d"}
Jul 18 01:21:31.853746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:31.853374    2567 scope.go:117] "RemoveContainer" containerID="fcd423e616a704638a6820de921b67a2acf2fe931e7f38657ce801e38953ca94"
Jul 18 01:21:31.853746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:31.853601    2567 scope.go:117] "RemoveContainer" containerID="13e4c2c2618c7823c02c9ff87b19e2ee03e63b1e50837329a88fffc8d299659d"
Jul 18 01:21:31.853915 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:31.853847    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx_odh-ai-gateway-infra(e40c4e4e-940a-482b-91d9-0ef8c39953a4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4"
Jul 18 01:21:32.860096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:32.860065    2567 scope.go:117] "RemoveContainer" containerID="13e4c2c2618c7823c02c9ff87b19e2ee03e63b1e50837329a88fffc8d299659d"
Jul 18 01:21:32.860620 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:32.860244    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx_odh-ai-gateway-infra(e40c4e4e-940a-482b-91d9-0ef8c39953a4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4"
Jul 18 01:21:35.978089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:35.978052    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"]
Jul 18 01:21:35.978513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:35.978497    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:35.978513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:35.978511    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:35.978596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:35.978576    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2cea0ac-8ee6-42ba-ac0c-e04e9f1eb011" containerName="revoke-keys"
Jul 18 01:21:35.982820 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:35.982803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:35.989169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:35.989102    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"]
Jul 18 01:21:36.063451 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.063402    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b129017-f060-4f9e-ae48-549621a7a1de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5\" (UID: \"3b129017-f060-4f9e-ae48-549621a7a1de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:36.164870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.164830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b129017-f060-4f9e-ae48-549621a7a1de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5\" (UID: \"3b129017-f060-4f9e-ae48-549621a7a1de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:36.165524 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.165505    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b129017-f060-4f9e-ae48-549621a7a1de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5\" (UID: \"3b129017-f060-4f9e-ae48-549621a7a1de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:36.295561 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.295466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:36.440086 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.440051    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"]
Jul 18 01:21:36.444446 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:21:36.444418    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b129017_f060_4f9e_ae48_549621a7a1de.slice/crio-b01a8476d0357a3f298bbcce94ffcde013159049854ea9fcdf61f63aae556914 WatchSource:0}: Error finding container b01a8476d0357a3f298bbcce94ffcde013159049854ea9fcdf61f63aae556914: Status 404 returned error can't find the container with id b01a8476d0357a3f298bbcce94ffcde013159049854ea9fcdf61f63aae556914
Jul 18 01:21:36.881006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.880971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" event={"ID":"3b129017-f060-4f9e-ae48-549621a7a1de","Type":"ContainerStarted","Data":"d1a7fcd39dfdb5b37cc059aa4e30cd6a738eb91fceac32719ab6f36300293c75"}
Jul 18 01:21:36.881180 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.881026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" event={"ID":"3b129017-f060-4f9e-ae48-549621a7a1de","Type":"ContainerStarted","Data":"b01a8476d0357a3f298bbcce94ffcde013159049854ea9fcdf61f63aae556914"}
Jul 18 01:21:36.896883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:36.896832    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" podStartSLOduration=1.896815052 podStartE2EDuration="1.896815052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:21:36.895549338 +0000 UTC m=+2476.423054301" watchObservedRunningTime="2026-07-18 01:21:36.896815052 +0000 UTC m=+2476.424320012"
Jul 18 01:21:37.887961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:37.887925    2567 generic.go:356] "Generic (PLEG): container finished" podID="3b129017-f060-4f9e-ae48-549621a7a1de" containerID="d1a7fcd39dfdb5b37cc059aa4e30cd6a738eb91fceac32719ab6f36300293c75" exitCode=7
Jul 18 01:21:37.888375 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:37.887999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" event={"ID":"3b129017-f060-4f9e-ae48-549621a7a1de","Type":"ContainerDied","Data":"d1a7fcd39dfdb5b37cc059aa4e30cd6a738eb91fceac32719ab6f36300293c75"}
Jul 18 01:21:37.888375 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:37.888172    2567 scope.go:117] "RemoveContainer" containerID="d1a7fcd39dfdb5b37cc059aa4e30cd6a738eb91fceac32719ab6f36300293c75"
Jul 18 01:21:38.895026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:38.894983    2567 generic.go:356] "Generic (PLEG): container finished" podID="3b129017-f060-4f9e-ae48-549621a7a1de" containerID="c81ed1de7c9d4c3e79d592ab95b6aec1a521203382512e90352cd574a24e5422" exitCode=7
Jul 18 01:21:38.895487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:38.895075    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" event={"ID":"3b129017-f060-4f9e-ae48-549621a7a1de","Type":"ContainerDied","Data":"c81ed1de7c9d4c3e79d592ab95b6aec1a521203382512e90352cd574a24e5422"}
Jul 18 01:21:38.895487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:38.895170    2567 scope.go:117] "RemoveContainer" containerID="d1a7fcd39dfdb5b37cc059aa4e30cd6a738eb91fceac32719ab6f36300293c75"
Jul 18 01:21:38.895487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:38.895486    2567 scope.go:117] "RemoveContainer" containerID="c81ed1de7c9d4c3e79d592ab95b6aec1a521203382512e90352cd574a24e5422"
Jul 18 01:21:38.895691 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:38.895671    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5_odh-ai-gateway-infra(3b129017-f060-4f9e-ae48-549621a7a1de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" podUID="3b129017-f060-4f9e-ae48-549621a7a1de"
Jul 18 01:21:39.901908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:39.901882    2567 scope.go:117] "RemoveContainer" containerID="c81ed1de7c9d4c3e79d592ab95b6aec1a521203382512e90352cd574a24e5422"
Jul 18 01:21:39.902311 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:39.902059    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5_odh-ai-gateway-infra(3b129017-f060-4f9e-ae48-549621a7a1de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" podUID="3b129017-f060-4f9e-ae48-549621a7a1de"
Jul 18 01:21:39.912929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:39.912909    2567 scope.go:117] "RemoveContainer" containerID="8883b403cee2014c34a4421e1a2a152e5565aa3d089cb3de1d2d88e727b4471f"
Jul 18 01:21:40.909331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:40.909297    2567 generic.go:356] "Generic (PLEG): container finished" podID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerID="4cf2c79f7764d74986f1f3e9c95de3ff3dc0d1363bca4f5ba7b34a6b13d0dc01" exitCode=6
Jul 18 01:21:40.909800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:40.909371    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" event={"ID":"988b847a-783b-43f7-bcd9-d921ce3e19fa","Type":"ContainerDied","Data":"4cf2c79f7764d74986f1f3e9c95de3ff3dc0d1363bca4f5ba7b34a6b13d0dc01"}
Jul 18 01:21:40.909800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:40.909432    2567 scope.go:117] "RemoveContainer" containerID="8883b403cee2014c34a4421e1a2a152e5565aa3d089cb3de1d2d88e727b4471f"
Jul 18 01:21:40.909800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:40.909702    2567 scope.go:117] "RemoveContainer" containerID="4cf2c79f7764d74986f1f3e9c95de3ff3dc0d1363bca4f5ba7b34a6b13d0dc01"
Jul 18 01:21:40.909955 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:40.909932    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh_odh-ai-gateway-infra(988b847a-783b-43f7-bcd9-d921ce3e19fa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa"
Jul 18 01:21:41.937743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:41.937709    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"]
Jul 18 01:21:42.083167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.083137    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:42.222473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.222370    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988b847a-783b-43f7-bcd9-d921ce3e19fa-maas-api-service-ca\") pod \"988b847a-783b-43f7-bcd9-d921ce3e19fa\" (UID: \"988b847a-783b-43f7-bcd9-d921ce3e19fa\") "
Jul 18 01:21:42.222771 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.222747    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/988b847a-783b-43f7-bcd9-d921ce3e19fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "988b847a-783b-43f7-bcd9-d921ce3e19fa" (UID: "988b847a-783b-43f7-bcd9-d921ce3e19fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:21:42.323375 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.323335    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/988b847a-783b-43f7-bcd9-d921ce3e19fa-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:42.932157 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.932121    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh" event={"ID":"988b847a-783b-43f7-bcd9-d921ce3e19fa","Type":"ContainerDied","Data":"ac9710775892606c070a59394bee4cf043aead9fd3eef4636e353a12fb06bc87"}
Jul 18 01:21:42.932314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.932141    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"
Jul 18 01:21:42.932314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.932216    2567 scope.go:117] "RemoveContainer" containerID="4cf2c79f7764d74986f1f3e9c95de3ff3dc0d1363bca4f5ba7b34a6b13d0dc01"
Jul 18 01:21:42.949589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.949550    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"]
Jul 18 01:21:42.954867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:42.954839    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-mlqxh"]
Jul 18 01:21:43.913132 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:43.913092    2567 scope.go:117] "RemoveContainer" containerID="13e4c2c2618c7823c02c9ff87b19e2ee03e63b1e50837329a88fffc8d299659d"
Jul 18 01:21:44.919486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:44.919405    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" path="/var/lib/kubelet/pods/988b847a-783b-43f7-bcd9-d921ce3e19fa/volumes"
Jul 18 01:21:44.945935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:44.945906    2567 generic.go:356] "Generic (PLEG): container finished" podID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerID="9cfa85215256076c0f97a3a0bf484ecb09bed16462980b5993642d2cd99d106d" exitCode=6
Jul 18 01:21:44.946103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:44.945986    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" event={"ID":"e40c4e4e-940a-482b-91d9-0ef8c39953a4","Type":"ContainerDied","Data":"9cfa85215256076c0f97a3a0bf484ecb09bed16462980b5993642d2cd99d106d"}
Jul 18 01:21:44.946103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:44.946053    2567 scope.go:117] "RemoveContainer" containerID="13e4c2c2618c7823c02c9ff87b19e2ee03e63b1e50837329a88fffc8d299659d"
Jul 18 01:21:44.946412 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:44.946399    2567 scope.go:117] "RemoveContainer" containerID="9cfa85215256076c0f97a3a0bf484ecb09bed16462980b5993642d2cd99d106d"
Jul 18 01:21:44.946641 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:44.946622    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx_odh-ai-gateway-infra(e40c4e4e-940a-482b-91d9-0ef8c39953a4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4"
Jul 18 01:21:45.969717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:45.969681    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"]
Jul 18 01:21:45.978081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:45.978050    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"]
Jul 18 01:21:45.978281 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:45.978244    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy" containerID="cri-o://f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d" gracePeriod=30
Jul 18 01:21:46.006325 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.006262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:46.092665 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.092638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:46.116440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.116405    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:46.260333 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.260232    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40c4e4e-940a-482b-91d9-0ef8c39953a4-maas-api-service-ca\") pod \"e40c4e4e-940a-482b-91d9-0ef8c39953a4\" (UID: \"e40c4e4e-940a-482b-91d9-0ef8c39953a4\") "
Jul 18 01:21:46.260591 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.260568    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e40c4e4e-940a-482b-91d9-0ef8c39953a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e40c4e4e-940a-482b-91d9-0ef8c39953a4" (UID: "e40c4e4e-940a-482b-91d9-0ef8c39953a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:21:46.361435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.361398    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e40c4e4e-940a-482b-91d9-0ef8c39953a4-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:46.959358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.959327    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"
Jul 18 01:21:46.959534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.959319    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx" event={"ID":"e40c4e4e-940a-482b-91d9-0ef8c39953a4","Type":"ContainerDied","Data":"3e2c620da995848d03c1fcef682d36cce470572094d23bff1c0712edba1dabfc"}
Jul 18 01:21:46.959534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.959456    2567 scope.go:117] "RemoveContainer" containerID="9cfa85215256076c0f97a3a0bf484ecb09bed16462980b5993642d2cd99d106d"
Jul 18 01:21:46.973895 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.973867    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"]
Jul 18 01:21:46.981793 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:46.981752    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-v5bgx"]
Jul 18 01:21:48.919474 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:48.919439    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" path="/var/lib/kubelet/pods/e40c4e4e-940a-482b-91d9-0ef8c39953a4/volumes"
Jul 18 01:21:49.004894 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:49.004856    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:21:51.237885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.237821    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:21:51.403970 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.403937    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istiod-ca-cert\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.403981    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-envoy\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404036    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-data\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404052    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-credential-socket\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404101    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-socket\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404130    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-token\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404160    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-podinfo\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404209    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-certs\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404307    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97hdg\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-kube-api-access-97hdg\") pod \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\" (UID: \"d0e2afd5-8888-48a8-aa8f-68136a737e2f\") "
Jul 18 01:21:51.404622 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404462    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:21:51.404622 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404477    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:21:51.404622 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404469    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:21:51.404877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404850    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.404877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404879    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.405238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.404893    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.405629 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.405600    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:21:51.405725 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.405676    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:21:51.406688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.406663    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:21:51.406787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.406750    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-kube-api-access-97hdg" (OuterVolumeSpecName: "kube-api-access-97hdg") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "kube-api-access-97hdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:21:51.407135 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.407113    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:21:51.407135 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.407121    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0e2afd5-8888-48a8-aa8f-68136a737e2f" (UID: "d0e2afd5-8888-48a8-aa8f-68136a737e2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:21:51.506241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.506138    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.506241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.506175    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.506241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.506188    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.506241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.506199    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97hdg\" (UniqueName: \"kubernetes.io/projected/d0e2afd5-8888-48a8-aa8f-68136a737e2f-kube-api-access-97hdg\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.506241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.506212    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.506241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.506223    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e2afd5-8888-48a8-aa8f-68136a737e2f-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:51.641242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.641214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:51.913376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.913342    2567 scope.go:117] "RemoveContainer" containerID="c81ed1de7c9d4c3e79d592ab95b6aec1a521203382512e90352cd574a24e5422"
Jul 18 01:21:51.989022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.988979    2567 generic.go:356] "Generic (PLEG): container finished" podID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerID="f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d" exitCode=0
Jul 18 01:21:51.989227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.989048    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"
Jul 18 01:21:51.989227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.989058    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" event={"ID":"d0e2afd5-8888-48a8-aa8f-68136a737e2f","Type":"ContainerDied","Data":"f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d"}
Jul 18 01:21:51.989227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.989108    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8" event={"ID":"d0e2afd5-8888-48a8-aa8f-68136a737e2f","Type":"ContainerDied","Data":"7a4b30d6d5b7839c177c7b3914f59a7fee18cbf6057e07ed7b3713ba4df4f95c"}
Jul 18 01:21:51.989227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:51.989130    2567 scope.go:117] "RemoveContainer" containerID="f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d"
Jul 18 01:21:52.005037 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.005015    2567 scope.go:117] "RemoveContainer" containerID="94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"
Jul 18 01:21:52.014524 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.014497    2567 scope.go:117] "RemoveContainer" containerID="f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d"
Jul 18 01:21:52.014828 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:52.014808    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d\": container with ID starting with f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d not found: ID does not exist" containerID="f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d"
Jul 18 01:21:52.014926 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.014837    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d"} err="failed to get container status \"f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d\": rpc error: code = NotFound desc = could not find container \"f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d\": container with ID starting with f88b5a4663322b8b825a6d6fe2cbd368fc67970f76f377d3deee110ba343422d not found: ID does not exist"
Jul 18 01:21:52.014926 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.014855    2567 scope.go:117] "RemoveContainer" containerID="94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"
Jul 18 01:21:52.015122 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:52.015106    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d\": container with ID starting with 94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d not found: ID does not exist" containerID="94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"
Jul 18 01:21:52.015188 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.015125    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d"} err="failed to get container status \"94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d\": rpc error: code = NotFound desc = could not find container \"94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d\": container with ID starting with 94dd4ef7ebcc118e950c04be977a3e790fc9eeb4ed9a4bae93fc8a8dd732385d not found: ID does not exist"
Jul 18 01:21:52.016178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.016155    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"]
Jul 18 01:21:52.019774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.019749    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8942d9e2-openshift-default-5bd855b799-ltfh8"]
Jul 18 01:21:52.070498 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.070469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:52.919565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.919470    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" path="/var/lib/kubelet/pods/d0e2afd5-8888-48a8-aa8f-68136a737e2f/volumes"
Jul 18 01:21:52.996716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.996679    2567 generic.go:356] "Generic (PLEG): container finished" podID="3b129017-f060-4f9e-ae48-549621a7a1de" containerID="246b36c0de1942230df93a8b4478412a02423939b8cc42ffeb6453f9afb21a5b" exitCode=6
Jul 18 01:21:52.996918 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.996758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" event={"ID":"3b129017-f060-4f9e-ae48-549621a7a1de","Type":"ContainerDied","Data":"246b36c0de1942230df93a8b4478412a02423939b8cc42ffeb6453f9afb21a5b"}
Jul 18 01:21:52.996918 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.996822    2567 scope.go:117] "RemoveContainer" containerID="c81ed1de7c9d4c3e79d592ab95b6aec1a521203382512e90352cd574a24e5422"
Jul 18 01:21:52.997161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:52.997136    2567 scope.go:117] "RemoveContainer" containerID="246b36c0de1942230df93a8b4478412a02423939b8cc42ffeb6453f9afb21a5b"
Jul 18 01:21:52.997425 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:21:52.997406    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5_odh-ai-gateway-infra(3b129017-f060-4f9e-ae48-549621a7a1de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" podUID="3b129017-f060-4f9e-ae48-549621a7a1de"
Jul 18 01:21:54.028519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:54.028479    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"]
Jul 18 01:21:54.169809 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:54.169783    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:54.332604 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:54.332564    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b129017-f060-4f9e-ae48-549621a7a1de-maas-api-service-ca\") pod \"3b129017-f060-4f9e-ae48-549621a7a1de\" (UID: \"3b129017-f060-4f9e-ae48-549621a7a1de\") "
Jul 18 01:21:54.332940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:54.332914    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b129017-f060-4f9e-ae48-549621a7a1de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b129017-f060-4f9e-ae48-549621a7a1de" (UID: "3b129017-f060-4f9e-ae48-549621a7a1de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:21:54.433959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:54.433910    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b129017-f060-4f9e-ae48-549621a7a1de-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:21:55.010584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:55.010553    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"
Jul 18 01:21:55.010761 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:55.010583    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5" event={"ID":"3b129017-f060-4f9e-ae48-549621a7a1de","Type":"ContainerDied","Data":"b01a8476d0357a3f298bbcce94ffcde013159049854ea9fcdf61f63aae556914"}
Jul 18 01:21:55.010761 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:55.010654    2567 scope.go:117] "RemoveContainer" containerID="246b36c0de1942230df93a8b4478412a02423939b8cc42ffeb6453f9afb21a5b"
Jul 18 01:21:55.025898 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:55.025854    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"]
Jul 18 01:21:55.030076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:55.030051    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz4x5"]
Jul 18 01:21:56.918790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:56.918758    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" path="/var/lib/kubelet/pods/3b129017-f060-4f9e-ae48-549621a7a1de/volumes"
Jul 18 01:21:57.531307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:57.531260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:57.641735 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:57.641702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:58.006357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:58.006327    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:58.109884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:58.109857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:58.207203 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:58.207165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:21:59.613012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:21:59.612981    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:03.422429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422396    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"]
Jul 18 01:22:03.422819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422775    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy"
Jul 18 01:22:03.422819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422787    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy"
Jul 18 01:22:03.422819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422801    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:03.422819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422807    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422827    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422836    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422845    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422850    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422856    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422861    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422866    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422871    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422877    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422883    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422893    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:03.422951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422900    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422974    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422985    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.422993    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.423002    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.423008    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.423015    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.423019    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e40c4e4e-940a-482b-91d9-0ef8c39953a4" containerName="revoke-keys"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.423026    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy"
Jul 18 01:22:03.423340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.423031    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:03.426153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.426137    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:03.428511 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.428490    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:22:03.428658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.428496    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:22:03.435168 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.435105    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"]
Jul 18 01:22:03.509367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.509331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4644d41-1153-463e-84aa-4d9aa8b6e07d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s\" (UID: \"f4644d41-1153-463e-84aa-4d9aa8b6e07d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:03.609855 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.609816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4644d41-1153-463e-84aa-4d9aa8b6e07d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s\" (UID: \"f4644d41-1153-463e-84aa-4d9aa8b6e07d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:03.610499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.610480    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4644d41-1153-463e-84aa-4d9aa8b6e07d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s\" (UID: \"f4644d41-1153-463e-84aa-4d9aa8b6e07d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:03.738121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.738034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:03.877922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:03.877893    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"]
Jul 18 01:22:03.881058 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:22:03.881029    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4644d41_1153_463e_84aa_4d9aa8b6e07d.slice/crio-35dffe4afe132d0a590a6487da9d483f07dcc2f3d26bf9f00dcdaf8b7969cf9e WatchSource:0}: Error finding container 35dffe4afe132d0a590a6487da9d483f07dcc2f3d26bf9f00dcdaf8b7969cf9e: Status 404 returned error can't find the container with id 35dffe4afe132d0a590a6487da9d483f07dcc2f3d26bf9f00dcdaf8b7969cf9e
Jul 18 01:22:04.058673 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:04.058642    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" event={"ID":"f4644d41-1153-463e-84aa-4d9aa8b6e07d","Type":"ContainerStarted","Data":"35dffe4afe132d0a590a6487da9d483f07dcc2f3d26bf9f00dcdaf8b7969cf9e"}
Jul 18 01:22:05.064832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.064796    2567 generic.go:356] "Generic (PLEG): container finished" podID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerID="6f88b36f60b78e926cd90733d9240cb4dbd02a2ad88e4ed2c77bcaba3e9bd8de" exitCode=6
Jul 18 01:22:05.065251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.064889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" event={"ID":"f4644d41-1153-463e-84aa-4d9aa8b6e07d","Type":"ContainerDied","Data":"6f88b36f60b78e926cd90733d9240cb4dbd02a2ad88e4ed2c77bcaba3e9bd8de"}
Jul 18 01:22:05.065251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.065095    2567 scope.go:117] "RemoveContainer" containerID="6f88b36f60b78e926cd90733d9240cb4dbd02a2ad88e4ed2c77bcaba3e9bd8de"
Jul 18 01:22:05.374079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.374052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:05.474376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.474346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:05.588638 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.588592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:05.701465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:05.701333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:06.064496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:06.064465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:06.071660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:06.071628    2567 generic.go:356] "Generic (PLEG): container finished" podID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerID="883c236517d0bda6af355edb60ea5c617b5548ca82a7f5fa3f4dfe99690a69ac" exitCode=6
Jul 18 01:22:06.072044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:06.071720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" event={"ID":"f4644d41-1153-463e-84aa-4d9aa8b6e07d","Type":"ContainerDied","Data":"883c236517d0bda6af355edb60ea5c617b5548ca82a7f5fa3f4dfe99690a69ac"}
Jul 18 01:22:06.072044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:06.071805    2567 scope.go:117] "RemoveContainer" containerID="6f88b36f60b78e926cd90733d9240cb4dbd02a2ad88e4ed2c77bcaba3e9bd8de"
Jul 18 01:22:06.072044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:06.071986    2567 scope.go:117] "RemoveContainer" containerID="883c236517d0bda6af355edb60ea5c617b5548ca82a7f5fa3f4dfe99690a69ac"
Jul 18 01:22:06.072249 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:06.072228    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s_odh-ai-gateway-infra(f4644d41-1153-463e-84aa-4d9aa8b6e07d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d"
Jul 18 01:22:07.078067 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:07.078041    2567 scope.go:117] "RemoveContainer" containerID="883c236517d0bda6af355edb60ea5c617b5548ca82a7f5fa3f4dfe99690a69ac"
Jul 18 01:22:07.078466 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:07.078221    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s_odh-ai-gateway-infra(f4644d41-1153-463e-84aa-4d9aa8b6e07d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d"
Jul 18 01:22:07.667621 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:07.667589    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:18.820580 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820491    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"]
Jul 18 01:22:18.820944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820872    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy"
Jul 18 01:22:18.820944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820883    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e2afd5-8888-48a8-aa8f-68136a737e2f" containerName="istio-proxy"
Jul 18 01:22:18.820944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820898    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:18.820944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820903    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:18.820944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820920    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:18.820944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820926    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:18.821159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820985    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b129017-f060-4f9e-ae48-549621a7a1de" containerName="revoke-keys"
Jul 18 01:22:18.821159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.820993    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="988b847a-783b-43f7-bcd9-d921ce3e19fa" containerName="revoke-keys"
Jul 18 01:22:18.825397 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.825376    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:18.833693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.833666    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"]
Jul 18 01:22:18.946031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:18.945991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/210b9aec-62de-4608-b288-44c429fb71f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf\" (UID: \"210b9aec-62de-4608-b288-44c429fb71f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:19.047495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:19.047459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/210b9aec-62de-4608-b288-44c429fb71f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf\" (UID: \"210b9aec-62de-4608-b288-44c429fb71f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:19.048154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:19.048130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/210b9aec-62de-4608-b288-44c429fb71f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf\" (UID: \"210b9aec-62de-4608-b288-44c429fb71f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:19.136979 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:19.136885    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:19.277211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:19.277180    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"]
Jul 18 01:22:19.279928 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:22:19.279901    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210b9aec_62de_4608_b288_44c429fb71f4.slice/crio-d41a720b82dac7e58ca6cc73c50b52fd0f756c07d41827fe6fe4409e4638c9cb WatchSource:0}: Error finding container d41a720b82dac7e58ca6cc73c50b52fd0f756c07d41827fe6fe4409e4638c9cb: Status 404 returned error can't find the container with id d41a720b82dac7e58ca6cc73c50b52fd0f756c07d41827fe6fe4409e4638c9cb
Jul 18 01:22:20.146868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.146780    2567 generic.go:356] "Generic (PLEG): container finished" podID="210b9aec-62de-4608-b288-44c429fb71f4" containerID="72d506dfad389800e4b1020821febda71b5bf39dce03e3b339531fe4c9567827" exitCode=6
Jul 18 01:22:20.146868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.146838    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" event={"ID":"210b9aec-62de-4608-b288-44c429fb71f4","Type":"ContainerDied","Data":"72d506dfad389800e4b1020821febda71b5bf39dce03e3b339531fe4c9567827"}
Jul 18 01:22:20.147308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.146880    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" event={"ID":"210b9aec-62de-4608-b288-44c429fb71f4","Type":"ContainerStarted","Data":"d41a720b82dac7e58ca6cc73c50b52fd0f756c07d41827fe6fe4409e4638c9cb"}
Jul 18 01:22:20.147308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.147077    2567 scope.go:117] "RemoveContainer" containerID="72d506dfad389800e4b1020821febda71b5bf39dce03e3b339531fe4c9567827"
Jul 18 01:22:20.959900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.959867    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"]
Jul 18 01:22:20.963432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.963414    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:20.970626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:20.970590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"]
Jul 18 01:22:21.065385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.065346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e09959-ad6c-46e3-8efd-abaf3b875b2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk\" (UID: \"71e09959-ad6c-46e3-8efd-abaf3b875b2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:21.153970 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.153934    2567 generic.go:356] "Generic (PLEG): container finished" podID="210b9aec-62de-4608-b288-44c429fb71f4" containerID="b59f6be5d1b44aa7a0eea4f6f635e74f4b88fa46849df3b9dbd1dccce6e2e928" exitCode=6
Jul 18 01:22:21.154463 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.154025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" event={"ID":"210b9aec-62de-4608-b288-44c429fb71f4","Type":"ContainerDied","Data":"b59f6be5d1b44aa7a0eea4f6f635e74f4b88fa46849df3b9dbd1dccce6e2e928"}
Jul 18 01:22:21.154463 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.154084    2567 scope.go:117] "RemoveContainer" containerID="72d506dfad389800e4b1020821febda71b5bf39dce03e3b339531fe4c9567827"
Jul 18 01:22:21.154463 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.154329    2567 scope.go:117] "RemoveContainer" containerID="b59f6be5d1b44aa7a0eea4f6f635e74f4b88fa46849df3b9dbd1dccce6e2e928"
Jul 18 01:22:21.154624 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:21.154562    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf_odh-ai-gateway-infra(210b9aec-62de-4608-b288-44c429fb71f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" podUID="210b9aec-62de-4608-b288-44c429fb71f4"
Jul 18 01:22:21.165824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.165801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e09959-ad6c-46e3-8efd-abaf3b875b2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk\" (UID: \"71e09959-ad6c-46e3-8efd-abaf3b875b2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:21.166358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.166341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e09959-ad6c-46e3-8efd-abaf3b875b2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk\" (UID: \"71e09959-ad6c-46e3-8efd-abaf3b875b2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:21.275479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.275442    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:21.419028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.418991    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"]
Jul 18 01:22:21.421131 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:22:21.421100    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e09959_ad6c_46e3_8efd_abaf3b875b2a.slice/crio-7fb2167c604707972a23450c836d6f31d0ce40a44bc0ecbff57e519073495c18 WatchSource:0}: Error finding container 7fb2167c604707972a23450c836d6f31d0ce40a44bc0ecbff57e519073495c18: Status 404 returned error can't find the container with id 7fb2167c604707972a23450c836d6f31d0ce40a44bc0ecbff57e519073495c18
Jul 18 01:22:21.644352 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.644231    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:21.704239 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.704193    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:21.913619 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:21.913539    2567 scope.go:117] "RemoveContainer" containerID="883c236517d0bda6af355edb60ea5c617b5548ca82a7f5fa3f4dfe99690a69ac"
Jul 18 01:22:22.160322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:22.160291    2567 generic.go:356] "Generic (PLEG): container finished" podID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerID="50f407e922f7d4aaf96941f733a008443972627f94bac5149190940ab897c6fd" exitCode=6
Jul 18 01:22:22.160710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:22.160379    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" event={"ID":"71e09959-ad6c-46e3-8efd-abaf3b875b2a","Type":"ContainerDied","Data":"50f407e922f7d4aaf96941f733a008443972627f94bac5149190940ab897c6fd"}
Jul 18 01:22:22.160710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:22.160432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" event={"ID":"71e09959-ad6c-46e3-8efd-abaf3b875b2a","Type":"ContainerStarted","Data":"7fb2167c604707972a23450c836d6f31d0ce40a44bc0ecbff57e519073495c18"}
Jul 18 01:22:22.160710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:22.160588    2567 scope.go:117] "RemoveContainer" containerID="50f407e922f7d4aaf96941f733a008443972627f94bac5149190940ab897c6fd"
Jul 18 01:22:22.162522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:22.162506    2567 scope.go:117] "RemoveContainer" containerID="b59f6be5d1b44aa7a0eea4f6f635e74f4b88fa46849df3b9dbd1dccce6e2e928"
Jul 18 01:22:22.162734 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:22.162695    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf_odh-ai-gateway-infra(210b9aec-62de-4608-b288-44c429fb71f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" podUID="210b9aec-62de-4608-b288-44c429fb71f4"
Jul 18 01:22:23.169046 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.169011    2567 generic.go:356] "Generic (PLEG): container finished" podID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerID="5c1277ab7ff5e27c09388fc0d703a8cf9fc45b5ccb0216fa2a4081d97f34dded" exitCode=6
Jul 18 01:22:23.169644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.169085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" event={"ID":"f4644d41-1153-463e-84aa-4d9aa8b6e07d","Type":"ContainerDied","Data":"5c1277ab7ff5e27c09388fc0d703a8cf9fc45b5ccb0216fa2a4081d97f34dded"}
Jul 18 01:22:23.169644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.169158    2567 scope.go:117] "RemoveContainer" containerID="883c236517d0bda6af355edb60ea5c617b5548ca82a7f5fa3f4dfe99690a69ac"
Jul 18 01:22:23.169644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.169472    2567 scope.go:117] "RemoveContainer" containerID="5c1277ab7ff5e27c09388fc0d703a8cf9fc45b5ccb0216fa2a4081d97f34dded"
Jul 18 01:22:23.169797 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:23.169717    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s_odh-ai-gateway-infra(f4644d41-1153-463e-84aa-4d9aa8b6e07d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d"
Jul 18 01:22:23.170988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.170966    2567 generic.go:356] "Generic (PLEG): container finished" podID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerID="d93a648f9b0e311ce44a74638b1c071022fe8c76c520f9c7108aad4cf1cdaa89" exitCode=6
Jul 18 01:22:23.171097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.171064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" event={"ID":"71e09959-ad6c-46e3-8efd-abaf3b875b2a","Type":"ContainerDied","Data":"d93a648f9b0e311ce44a74638b1c071022fe8c76c520f9c7108aad4cf1cdaa89"}
Jul 18 01:22:23.171239 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.171220    2567 scope.go:117] "RemoveContainer" containerID="d93a648f9b0e311ce44a74638b1c071022fe8c76c520f9c7108aad4cf1cdaa89"
Jul 18 01:22:23.171477 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:23.171455    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk_odh-ai-gateway-infra(71e09959-ad6c-46e3-8efd-abaf3b875b2a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a"
Jul 18 01:22:23.180348 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:23.180326    2567 scope.go:117] "RemoveContainer" containerID="50f407e922f7d4aaf96941f733a008443972627f94bac5149190940ab897c6fd"
Jul 18 01:22:24.179413 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:24.179385    2567 scope.go:117] "RemoveContainer" containerID="d93a648f9b0e311ce44a74638b1c071022fe8c76c520f9c7108aad4cf1cdaa89"
Jul 18 01:22:24.179834 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:24.179576    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk_odh-ai-gateway-infra(71e09959-ad6c-46e3-8efd-abaf3b875b2a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a"
Jul 18 01:22:24.193816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:24.193773    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"]
Jul 18 01:22:24.339672 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:24.339645    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:24.396170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:24.396135    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4644d41-1153-463e-84aa-4d9aa8b6e07d-maas-api-service-ca\") pod \"f4644d41-1153-463e-84aa-4d9aa8b6e07d\" (UID: \"f4644d41-1153-463e-84aa-4d9aa8b6e07d\") "
Jul 18 01:22:24.396544 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:24.396519    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4644d41-1153-463e-84aa-4d9aa8b6e07d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4644d41-1153-463e-84aa-4d9aa8b6e07d" (UID: "f4644d41-1153-463e-84aa-4d9aa8b6e07d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:22:24.497759 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:24.497665    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4644d41-1153-463e-84aa-4d9aa8b6e07d-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:22:25.185935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:25.185848    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"
Jul 18 01:22:25.185935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:25.185890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s" event={"ID":"f4644d41-1153-463e-84aa-4d9aa8b6e07d","Type":"ContainerDied","Data":"35dffe4afe132d0a590a6487da9d483f07dcc2f3d26bf9f00dcdaf8b7969cf9e"}
Jul 18 01:22:25.186448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:25.185957    2567 scope.go:117] "RemoveContainer" containerID="5c1277ab7ff5e27c09388fc0d703a8cf9fc45b5ccb0216fa2a4081d97f34dded"
Jul 18 01:22:25.201987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:25.201960    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"]
Jul 18 01:22:25.208347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:25.208316    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-vhz8s"]
Jul 18 01:22:26.024456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024420    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"]
Jul 18 01:22:26.024807 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024794    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.024852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024808    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.024852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024816    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.024852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024822    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.024852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024836    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.024852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024842    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.025005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024902    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.025005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.024909    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:22:26.027837 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.027819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:26.036880 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.036804    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"]
Jul 18 01:22:26.111251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.111213    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c8db61-bc53-4c5a-b58e-19c11676a27d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m\" (UID: \"83c8db61-bc53-4c5a-b58e-19c11676a27d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:26.212783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.212747    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c8db61-bc53-4c5a-b58e-19c11676a27d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m\" (UID: \"83c8db61-bc53-4c5a-b58e-19c11676a27d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:26.213454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.213435    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c8db61-bc53-4c5a-b58e-19c11676a27d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m\" (UID: \"83c8db61-bc53-4c5a-b58e-19c11676a27d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:26.339486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.339435    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:26.486686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.486655    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"]
Jul 18 01:22:26.488644 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:22:26.488614    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c8db61_bc53_4c5a_b58e_19c11676a27d.slice/crio-1aed17cc10104282068c3eaeeababbe24ffae862f29177b52afa507200bbeba2 WatchSource:0}: Error finding container 1aed17cc10104282068c3eaeeababbe24ffae862f29177b52afa507200bbeba2: Status 404 returned error can't find the container with id 1aed17cc10104282068c3eaeeababbe24ffae862f29177b52afa507200bbeba2
Jul 18 01:22:26.920683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:26.920601    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" path="/var/lib/kubelet/pods/f4644d41-1153-463e-84aa-4d9aa8b6e07d/volumes"
Jul 18 01:22:27.200367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:27.200251    2567 generic.go:356] "Generic (PLEG): container finished" podID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerID="825493f63e7b827c102a732dddb80fda1d1cd0f51a98c11bdd00f45b48092d62" exitCode=6
Jul 18 01:22:27.200367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:27.200321    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" event={"ID":"83c8db61-bc53-4c5a-b58e-19c11676a27d","Type":"ContainerDied","Data":"825493f63e7b827c102a732dddb80fda1d1cd0f51a98c11bdd00f45b48092d62"}
Jul 18 01:22:27.200367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:27.200365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" event={"ID":"83c8db61-bc53-4c5a-b58e-19c11676a27d","Type":"ContainerStarted","Data":"1aed17cc10104282068c3eaeeababbe24ffae862f29177b52afa507200bbeba2"}
Jul 18 01:22:27.200607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:27.200571    2567 scope.go:117] "RemoveContainer" containerID="825493f63e7b827c102a732dddb80fda1d1cd0f51a98c11bdd00f45b48092d62"
Jul 18 01:22:28.207611 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:28.207574    2567 generic.go:356] "Generic (PLEG): container finished" podID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerID="a991c9ea041fec630804e8e7df6dd92fdb649fc5f18074cb6a398af1eb6b1f7d" exitCode=6
Jul 18 01:22:28.208109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:28.207618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" event={"ID":"83c8db61-bc53-4c5a-b58e-19c11676a27d","Type":"ContainerDied","Data":"a991c9ea041fec630804e8e7df6dd92fdb649fc5f18074cb6a398af1eb6b1f7d"}
Jul 18 01:22:28.208109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:28.207680    2567 scope.go:117] "RemoveContainer" containerID="825493f63e7b827c102a732dddb80fda1d1cd0f51a98c11bdd00f45b48092d62"
Jul 18 01:22:28.208109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:28.207903    2567 scope.go:117] "RemoveContainer" containerID="a991c9ea041fec630804e8e7df6dd92fdb649fc5f18074cb6a398af1eb6b1f7d"
Jul 18 01:22:28.208323 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:28.208167    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m_odh-ai-gateway-infra(83c8db61-bc53-4c5a-b58e-19c11676a27d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d"
Jul 18 01:22:29.214050 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:29.214024    2567 scope.go:117] "RemoveContainer" containerID="a991c9ea041fec630804e8e7df6dd92fdb649fc5f18074cb6a398af1eb6b1f7d"
Jul 18 01:22:29.214474 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:29.214213    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m_odh-ai-gateway-infra(83c8db61-bc53-4c5a-b58e-19c11676a27d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d"
Jul 18 01:22:31.994440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:31.994407    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:32.913296 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:32.913247    2567 scope.go:117] "RemoveContainer" containerID="b59f6be5d1b44aa7a0eea4f6f635e74f4b88fa46849df3b9dbd1dccce6e2e928"
Jul 18 01:22:33.237290 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:33.237187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" event={"ID":"210b9aec-62de-4608-b288-44c429fb71f4","Type":"ContainerStarted","Data":"37086af97bc1361db22be8bab1c258374533a71b6f9983264b09e91b91525b69"}
Jul 18 01:22:33.254371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:33.254319    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" podStartSLOduration=15.254304340000001 podStartE2EDuration="15.25430434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:22:33.251966755 +0000 UTC m=+2532.779471717" watchObservedRunningTime="2026-07-18 01:22:33.25430434 +0000 UTC m=+2532.781809299"
Jul 18 01:22:34.244532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:34.244502    2567 generic.go:356] "Generic (PLEG): container finished" podID="210b9aec-62de-4608-b288-44c429fb71f4" containerID="37086af97bc1361db22be8bab1c258374533a71b6f9983264b09e91b91525b69" exitCode=6
Jul 18 01:22:34.245049 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:34.244542    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" event={"ID":"210b9aec-62de-4608-b288-44c429fb71f4","Type":"ContainerDied","Data":"37086af97bc1361db22be8bab1c258374533a71b6f9983264b09e91b91525b69"}
Jul 18 01:22:34.245049 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:34.244591    2567 scope.go:117] "RemoveContainer" containerID="b59f6be5d1b44aa7a0eea4f6f635e74f4b88fa46849df3b9dbd1dccce6e2e928"
Jul 18 01:22:34.245049 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:34.244906    2567 scope.go:117] "RemoveContainer" containerID="37086af97bc1361db22be8bab1c258374533a71b6f9983264b09e91b91525b69"
Jul 18 01:22:34.245209 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:34.245161    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf_odh-ai-gateway-infra(210b9aec-62de-4608-b288-44c429fb71f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" podUID="210b9aec-62de-4608-b288-44c429fb71f4"
Jul 18 01:22:34.267676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:34.267650    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"]
Jul 18 01:22:35.391654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:35.391630    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:35.498005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:35.497966    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/210b9aec-62de-4608-b288-44c429fb71f4-maas-api-service-ca\") pod \"210b9aec-62de-4608-b288-44c429fb71f4\" (UID: \"210b9aec-62de-4608-b288-44c429fb71f4\") "
Jul 18 01:22:35.498354 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:35.498333    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210b9aec-62de-4608-b288-44c429fb71f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "210b9aec-62de-4608-b288-44c429fb71f4" (UID: "210b9aec-62de-4608-b288-44c429fb71f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:22:35.599461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:35.599406    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/210b9aec-62de-4608-b288-44c429fb71f4-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:22:36.257712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:36.257680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf" event={"ID":"210b9aec-62de-4608-b288-44c429fb71f4","Type":"ContainerDied","Data":"d41a720b82dac7e58ca6cc73c50b52fd0f756c07d41827fe6fe4409e4638c9cb"}
Jul 18 01:22:36.257712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:36.257700    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"
Jul 18 01:22:36.257993 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:36.257741    2567 scope.go:117] "RemoveContainer" containerID="37086af97bc1361db22be8bab1c258374533a71b6f9983264b09e91b91525b69"
Jul 18 01:22:36.279510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:36.279478    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"]
Jul 18 01:22:36.288344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:36.288311    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-znwvf"]
Jul 18 01:22:36.919280 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:36.919246    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="210b9aec-62de-4608-b288-44c429fb71f4" path="/var/lib/kubelet/pods/210b9aec-62de-4608-b288-44c429fb71f4/volumes"
Jul 18 01:22:37.616101 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:37.616074    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:37.718738 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:37.718709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:37.817705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:37.817668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:37.913402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:37.913319    2567 scope.go:117] "RemoveContainer" containerID="d93a648f9b0e311ce44a74638b1c071022fe8c76c520f9c7108aad4cf1cdaa89"
Jul 18 01:22:38.271936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:38.271905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" event={"ID":"71e09959-ad6c-46e3-8efd-abaf3b875b2a","Type":"ContainerStarted","Data":"4896e9675ca3ddcddd053319604310901748559265d8f0c8f7c2763ad7379dde"}
Jul 18 01:22:38.288022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:38.287971    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" podStartSLOduration=18.287955857 podStartE2EDuration="18.287955857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:22:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:22:38.28674039 +0000 UTC m=+2537.814245349" watchObservedRunningTime="2026-07-18 01:22:38.287955857 +0000 UTC m=+2537.815460816"
Jul 18 01:22:39.278801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:39.278767    2567 generic.go:356] "Generic (PLEG): container finished" podID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerID="4896e9675ca3ddcddd053319604310901748559265d8f0c8f7c2763ad7379dde" exitCode=6
Jul 18 01:22:39.279322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:39.278826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" event={"ID":"71e09959-ad6c-46e3-8efd-abaf3b875b2a","Type":"ContainerDied","Data":"4896e9675ca3ddcddd053319604310901748559265d8f0c8f7c2763ad7379dde"}
Jul 18 01:22:39.279322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:39.278876    2567 scope.go:117] "RemoveContainer" containerID="d93a648f9b0e311ce44a74638b1c071022fe8c76c520f9c7108aad4cf1cdaa89"
Jul 18 01:22:39.279322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:39.279215    2567 scope.go:117] "RemoveContainer" containerID="4896e9675ca3ddcddd053319604310901748559265d8f0c8f7c2763ad7379dde"
Jul 18 01:22:39.279535 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:39.279512    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk_odh-ai-gateway-infra(71e09959-ad6c-46e3-8efd-abaf3b875b2a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a"
Jul 18 01:22:39.297702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:39.297675    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"]
Jul 18 01:22:39.482396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:39.482358    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:40.437654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:40.437631    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:40.551039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:40.550952    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e09959-ad6c-46e3-8efd-abaf3b875b2a-maas-api-service-ca\") pod \"71e09959-ad6c-46e3-8efd-abaf3b875b2a\" (UID: \"71e09959-ad6c-46e3-8efd-abaf3b875b2a\") "
Jul 18 01:22:40.551340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:40.551318    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e09959-ad6c-46e3-8efd-abaf3b875b2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71e09959-ad6c-46e3-8efd-abaf3b875b2a" (UID: "71e09959-ad6c-46e3-8efd-abaf3b875b2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:22:40.652663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:40.652621    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e09959-ad6c-46e3-8efd-abaf3b875b2a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:22:40.915361 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:40.915326    2567 scope.go:117] "RemoveContainer" containerID="a991c9ea041fec630804e8e7df6dd92fdb649fc5f18074cb6a398af1eb6b1f7d"
Jul 18 01:22:41.293022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.292925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" event={"ID":"83c8db61-bc53-4c5a-b58e-19c11676a27d","Type":"ContainerStarted","Data":"b545889ceca6a192bfbb9f595ebcaa3a88cbb82b2d37d43a073e4594fcf7b59e"}
Jul 18 01:22:41.294094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.294074    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"
Jul 18 01:22:41.294217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.294109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk" event={"ID":"71e09959-ad6c-46e3-8efd-abaf3b875b2a","Type":"ContainerDied","Data":"7fb2167c604707972a23450c836d6f31d0ce40a44bc0ecbff57e519073495c18"}
Jul 18 01:22:41.294217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.294162    2567 scope.go:117] "RemoveContainer" containerID="4896e9675ca3ddcddd053319604310901748559265d8f0c8f7c2763ad7379dde"
Jul 18 01:22:41.313688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.313642    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" podStartSLOduration=15.313624605 podStartE2EDuration="15.313624605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:22:41.312665058 +0000 UTC m=+2540.840170017" watchObservedRunningTime="2026-07-18 01:22:41.313624605 +0000 UTC m=+2540.841129564"
Jul 18 01:22:41.328779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.328746    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"]
Jul 18 01:22:41.332552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:41.332512    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-2wdgk"]
Jul 18 01:22:42.302293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:42.302240    2567 generic.go:356] "Generic (PLEG): container finished" podID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerID="b545889ceca6a192bfbb9f595ebcaa3a88cbb82b2d37d43a073e4594fcf7b59e" exitCode=6
Jul 18 01:22:42.302732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:42.302313    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" event={"ID":"83c8db61-bc53-4c5a-b58e-19c11676a27d","Type":"ContainerDied","Data":"b545889ceca6a192bfbb9f595ebcaa3a88cbb82b2d37d43a073e4594fcf7b59e"}
Jul 18 01:22:42.302732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:42.302373    2567 scope.go:117] "RemoveContainer" containerID="a991c9ea041fec630804e8e7df6dd92fdb649fc5f18074cb6a398af1eb6b1f7d"
Jul 18 01:22:42.302732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:42.302670    2567 scope.go:117] "RemoveContainer" containerID="b545889ceca6a192bfbb9f595ebcaa3a88cbb82b2d37d43a073e4594fcf7b59e"
Jul 18 01:22:42.302929 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:22:42.302904    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m_odh-ai-gateway-infra(83c8db61-bc53-4c5a-b58e-19c11676a27d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d"
Jul 18 01:22:42.320989 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:42.320953    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"]
Jul 18 01:22:42.919662 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:42.919629    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" path="/var/lib/kubelet/pods/71e09959-ad6c-46e3-8efd-abaf3b875b2a/volumes"
Jul 18 01:22:43.450514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:43.450485    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:43.579439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:43.579347    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c8db61-bc53-4c5a-b58e-19c11676a27d-maas-api-service-ca\") pod \"83c8db61-bc53-4c5a-b58e-19c11676a27d\" (UID: \"83c8db61-bc53-4c5a-b58e-19c11676a27d\") "
Jul 18 01:22:43.579710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:43.579685    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83c8db61-bc53-4c5a-b58e-19c11676a27d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "83c8db61-bc53-4c5a-b58e-19c11676a27d" (UID: "83c8db61-bc53-4c5a-b58e-19c11676a27d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:22:43.680093 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:43.680064    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c8db61-bc53-4c5a-b58e-19c11676a27d-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:22:44.315092 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:44.315058    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"
Jul 18 01:22:44.315254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:44.315103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m" event={"ID":"83c8db61-bc53-4c5a-b58e-19c11676a27d","Type":"ContainerDied","Data":"1aed17cc10104282068c3eaeeababbe24ffae862f29177b52afa507200bbeba2"}
Jul 18 01:22:44.315254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:44.315164    2567 scope.go:117] "RemoveContainer" containerID="b545889ceca6a192bfbb9f595ebcaa3a88cbb82b2d37d43a073e4594fcf7b59e"
Jul 18 01:22:44.338018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:44.337989    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"]
Jul 18 01:22:44.343507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:44.343481    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-fck2m"]
Jul 18 01:22:44.918828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:44.918787    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" path="/var/lib/kubelet/pods/83c8db61-bc53-4c5a-b58e-19c11676a27d/volumes"
Jul 18 01:22:53.630024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:53.629988    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:59.624218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:59.624171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:59.685599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:59.685545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:59.767398 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:59.767363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:22:59.866065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:22:59.866012    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:03.462828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.462793    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"]
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463142    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463152    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463160    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463167    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463177    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463182    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463188    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463193    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463198    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463204    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463218    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463223    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463308    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463318    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463324    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463330    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463337    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463342    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463347    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463353    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463358    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4644d41-1153-463e-84aa-4d9aa8b6e07d" containerName="revoke-keys"
Jul 18 01:23:03.463653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.463363    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:03.466332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.466314    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:03.468680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.468655    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:23:03.468826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.468655    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:23:03.474642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.474621    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"]
Jul 18 01:23:03.538627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.538585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd662b46-608b-471f-929c-ec78f3f58cfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-whbbw\" (UID: \"fd662b46-608b-471f-929c-ec78f3f58cfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:03.640055 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.640016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd662b46-608b-471f-929c-ec78f3f58cfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-whbbw\" (UID: \"fd662b46-608b-471f-929c-ec78f3f58cfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:03.640653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.640633    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd662b46-608b-471f-929c-ec78f3f58cfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-whbbw\" (UID: \"fd662b46-608b-471f-929c-ec78f3f58cfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:03.778508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.778409    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:03.934021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:03.933992    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"]
Jul 18 01:23:03.936156 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:03.936121    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd662b46_608b_471f_929c_ec78f3f58cfe.slice/crio-3048b6ced9b8fccb27ce8b8660f73e68cf303e649daf1cd518faf2011a73fc98 WatchSource:0}: Error finding container 3048b6ced9b8fccb27ce8b8660f73e68cf303e649daf1cd518faf2011a73fc98: Status 404 returned error can't find the container with id 3048b6ced9b8fccb27ce8b8660f73e68cf303e649daf1cd518faf2011a73fc98
Jul 18 01:23:04.420179 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:04.420086    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerStarted","Data":"60097a7a0e2381661c75409e7fab5ec2513bfbf59d8c8cc7131c2c8e84b98fec"}
Jul 18 01:23:04.420179 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:04.420149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerStarted","Data":"3048b6ced9b8fccb27ce8b8660f73e68cf303e649daf1cd518faf2011a73fc98"}
Jul 18 01:23:04.435158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:04.435102    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" podStartSLOduration=1.4350857399999999 podStartE2EDuration="1.43508574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:23:04.433393306 +0000 UTC m=+2563.960898270" watchObservedRunningTime="2026-07-18 01:23:04.43508574 +0000 UTC m=+2563.962590700"
Jul 18 01:23:05.428121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:05.428086    2567 generic.go:356] "Generic (PLEG): container finished" podID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerID="60097a7a0e2381661c75409e7fab5ec2513bfbf59d8c8cc7131c2c8e84b98fec" exitCode=6
Jul 18 01:23:05.428585 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:05.428133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerDied","Data":"60097a7a0e2381661c75409e7fab5ec2513bfbf59d8c8cc7131c2c8e84b98fec"}
Jul 18 01:23:05.428585 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:05.428368    2567 scope.go:117] "RemoveContainer" containerID="60097a7a0e2381661c75409e7fab5ec2513bfbf59d8c8cc7131c2c8e84b98fec"
Jul 18 01:23:06.434862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:06.434831    2567 generic.go:356] "Generic (PLEG): container finished" podID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerID="1c1c7c957b73407cf457ace62839621e7c5d9ebbeed7aedd18fa4505f8efa217" exitCode=6
Jul 18 01:23:06.435308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:06.434909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerDied","Data":"1c1c7c957b73407cf457ace62839621e7c5d9ebbeed7aedd18fa4505f8efa217"}
Jul 18 01:23:06.435308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:06.434951    2567 scope.go:117] "RemoveContainer" containerID="60097a7a0e2381661c75409e7fab5ec2513bfbf59d8c8cc7131c2c8e84b98fec"
Jul 18 01:23:06.435308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:06.435155    2567 scope.go:117] "RemoveContainer" containerID="1c1c7c957b73407cf457ace62839621e7c5d9ebbeed7aedd18fa4505f8efa217"
Jul 18 01:23:06.435442 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:06.435386    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-whbbw_odh-ai-gateway-infra(fd662b46-608b-471f-929c-ec78f3f58cfe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe"
Jul 18 01:23:07.441644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:07.441613    2567 scope.go:117] "RemoveContainer" containerID="1c1c7c957b73407cf457ace62839621e7c5d9ebbeed7aedd18fa4505f8efa217"
Jul 18 01:23:07.442121 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:07.441848    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-whbbw_odh-ai-gateway-infra(fd662b46-608b-471f-929c-ec78f3f58cfe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe"
Jul 18 01:23:08.861583 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.861545    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"]
Jul 18 01:23:08.862127 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.862106    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:08.862211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.862129    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="210b9aec-62de-4608-b288-44c429fb71f4" containerName="revoke-keys"
Jul 18 01:23:08.862211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.862163    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:08.862211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.862173    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c8db61-bc53-4c5a-b58e-19c11676a27d" containerName="revoke-keys"
Jul 18 01:23:08.862211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.862189    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:08.862211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.862199    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e09959-ad6c-46e3-8efd-abaf3b875b2a" containerName="revoke-keys"
Jul 18 01:23:08.866493 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.866471    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:08.871535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.871510    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"]
Jul 18 01:23:08.996524 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:08.996484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a8dc52-957f-46da-83f1-3e523cd08a7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z\" (UID: \"58a8dc52-957f-46da-83f1-3e523cd08a7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:09.097808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:09.097770    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a8dc52-957f-46da-83f1-3e523cd08a7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z\" (UID: \"58a8dc52-957f-46da-83f1-3e523cd08a7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:09.098417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:09.098397    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a8dc52-957f-46da-83f1-3e523cd08a7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z\" (UID: \"58a8dc52-957f-46da-83f1-3e523cd08a7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:09.178855 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:09.178763    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:09.325008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:09.324950    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"]
Jul 18 01:23:09.331711 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:09.331679    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a8dc52_957f_46da_83f1_3e523cd08a7b.slice/crio-63abc14923e17914c6623ff90510b5f235e35851f3a8c7d069c3429a185bcd49 WatchSource:0}: Error finding container 63abc14923e17914c6623ff90510b5f235e35851f3a8c7d069c3429a185bcd49: Status 404 returned error can't find the container with id 63abc14923e17914c6623ff90510b5f235e35851f3a8c7d069c3429a185bcd49
Jul 18 01:23:09.453382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:09.453295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" event={"ID":"58a8dc52-957f-46da-83f1-3e523cd08a7b","Type":"ContainerStarted","Data":"63abc14923e17914c6623ff90510b5f235e35851f3a8c7d069c3429a185bcd49"}
Jul 18 01:23:10.349991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:10.349912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:10.459900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:10.459869    2567 generic.go:356] "Generic (PLEG): container finished" podID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerID="a8ba9bbf28d7a925d90c020915bdbd41ffa7284cf1eb0640c482f5b819e949e6" exitCode=6
Jul 18 01:23:10.460054 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:10.459941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" event={"ID":"58a8dc52-957f-46da-83f1-3e523cd08a7b","Type":"ContainerDied","Data":"a8ba9bbf28d7a925d90c020915bdbd41ffa7284cf1eb0640c482f5b819e949e6"}
Jul 18 01:23:10.460162 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:10.460150    2567 scope.go:117] "RemoveContainer" containerID="a8ba9bbf28d7a925d90c020915bdbd41ffa7284cf1eb0640c482f5b819e949e6"
Jul 18 01:23:11.124455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:11.124422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:11.467135 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:11.467101    2567 generic.go:356] "Generic (PLEG): container finished" podID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerID="7170a1f94325a88d71aaba33f00def795f3dc57fce8a058b40601dbf54c416c9" exitCode=6
Jul 18 01:23:11.467576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:11.467191    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" event={"ID":"58a8dc52-957f-46da-83f1-3e523cd08a7b","Type":"ContainerDied","Data":"7170a1f94325a88d71aaba33f00def795f3dc57fce8a058b40601dbf54c416c9"}
Jul 18 01:23:11.467576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:11.467257    2567 scope.go:117] "RemoveContainer" containerID="a8ba9bbf28d7a925d90c020915bdbd41ffa7284cf1eb0640c482f5b819e949e6"
Jul 18 01:23:11.467576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:11.467467    2567 scope.go:117] "RemoveContainer" containerID="7170a1f94325a88d71aaba33f00def795f3dc57fce8a058b40601dbf54c416c9"
Jul 18 01:23:11.467703 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:11.467691    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z_odh-ai-gateway-infra(58a8dc52-957f-46da-83f1-3e523cd08a7b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b"
Jul 18 01:23:12.473704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:12.473675    2567 scope.go:117] "RemoveContainer" containerID="7170a1f94325a88d71aaba33f00def795f3dc57fce8a058b40601dbf54c416c9"
Jul 18 01:23:12.474099 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:12.473882    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z_odh-ai-gateway-infra(58a8dc52-957f-46da-83f1-3e523cd08a7b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b"
Jul 18 01:23:16.063995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.063956    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"]
Jul 18 01:23:16.068654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.068632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:16.074657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.074623    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"]
Jul 18 01:23:16.161108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.161066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fab7f27-4a69-422b-a46b-80f1781beb13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-z2685\" (UID: \"6fab7f27-4a69-422b-a46b-80f1781beb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:16.262383 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.262340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fab7f27-4a69-422b-a46b-80f1781beb13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-z2685\" (UID: \"6fab7f27-4a69-422b-a46b-80f1781beb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:16.262966 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.262943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fab7f27-4a69-422b-a46b-80f1781beb13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-z2685\" (UID: \"6fab7f27-4a69-422b-a46b-80f1781beb13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:16.382373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.382258    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:16.522570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.522444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"]
Jul 18 01:23:16.525444 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:16.525418    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fab7f27_4a69_422b_a46b_80f1781beb13.slice/crio-68b9f76b4baadfd11775ebb105143bde8708a937b1de1f89b78198581e49d576 WatchSource:0}: Error finding container 68b9f76b4baadfd11775ebb105143bde8708a937b1de1f89b78198581e49d576: Status 404 returned error can't find the container with id 68b9f76b4baadfd11775ebb105143bde8708a937b1de1f89b78198581e49d576
Jul 18 01:23:16.792294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.792244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:16.849224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.849194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:16.934106 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:16.934027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:17.033503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.033473    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:17.133674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.133639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:17.502045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.502014    2567 generic.go:356] "Generic (PLEG): container finished" podID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerID="0df358c4016f9c89481eb6bfc5d0bb6991d2688004969d4d6f32fb684c9f161b" exitCode=6
Jul 18 01:23:17.502234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.502106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" event={"ID":"6fab7f27-4a69-422b-a46b-80f1781beb13","Type":"ContainerDied","Data":"0df358c4016f9c89481eb6bfc5d0bb6991d2688004969d4d6f32fb684c9f161b"}
Jul 18 01:23:17.502234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.502165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" event={"ID":"6fab7f27-4a69-422b-a46b-80f1781beb13","Type":"ContainerStarted","Data":"68b9f76b4baadfd11775ebb105143bde8708a937b1de1f89b78198581e49d576"}
Jul 18 01:23:17.502333 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.502309    2567 scope.go:117] "RemoveContainer" containerID="0df358c4016f9c89481eb6bfc5d0bb6991d2688004969d4d6f32fb684c9f161b"
Jul 18 01:23:17.843735 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.843704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:17.948755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:17.948722    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:18.045874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:18.045790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:18.508957 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:18.508923    2567 generic.go:356] "Generic (PLEG): container finished" podID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerID="84bca1ce741250c8616e630ed0d2f7424b3900f59cacd44458dcac4ddef76fda" exitCode=6
Jul 18 01:23:18.509417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:18.509011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" event={"ID":"6fab7f27-4a69-422b-a46b-80f1781beb13","Type":"ContainerDied","Data":"84bca1ce741250c8616e630ed0d2f7424b3900f59cacd44458dcac4ddef76fda"}
Jul 18 01:23:18.509417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:18.509077    2567 scope.go:117] "RemoveContainer" containerID="0df358c4016f9c89481eb6bfc5d0bb6991d2688004969d4d6f32fb684c9f161b"
Jul 18 01:23:18.509417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:18.509214    2567 scope.go:117] "RemoveContainer" containerID="84bca1ce741250c8616e630ed0d2f7424b3900f59cacd44458dcac4ddef76fda"
Jul 18 01:23:18.509563 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:18.509489    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-z2685_odh-ai-gateway-infra(6fab7f27-4a69-422b-a46b-80f1781beb13)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13"
Jul 18 01:23:19.516481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:19.516453    2567 scope.go:117] "RemoveContainer" containerID="84bca1ce741250c8616e630ed0d2f7424b3900f59cacd44458dcac4ddef76fda"
Jul 18 01:23:19.516870 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:19.516655    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-z2685_odh-ai-gateway-infra(6fab7f27-4a69-422b-a46b-80f1781beb13)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13"
Jul 18 01:23:19.683238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:19.683196    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:20.208701 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:20.208667    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:23:21.001673 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.001631    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"]
Jul 18 01:23:21.006450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.006428    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:21.010950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.010918    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"]
Jul 18 01:23:21.107589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.107550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e8a57-35b6-43aa-81c3-73c4e9c19859-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4\" (UID: \"ed1e8a57-35b6-43aa-81c3-73c4e9c19859\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:21.209035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.209004    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e8a57-35b6-43aa-81c3-73c4e9c19859-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4\" (UID: \"ed1e8a57-35b6-43aa-81c3-73c4e9c19859\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:21.209649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.209631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e8a57-35b6-43aa-81c3-73c4e9c19859-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4\" (UID: \"ed1e8a57-35b6-43aa-81c3-73c4e9c19859\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:21.319609 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.319577    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:21.465633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.465608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"]
Jul 18 01:23:21.467668 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:21.467638    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1e8a57_35b6_43aa_81c3_73c4e9c19859.slice/crio-095acec6f0383730e36a5541ca1f881b62dd2302a9a35cf43b4e5e1c223e8e2f WatchSource:0}: Error finding container 095acec6f0383730e36a5541ca1f881b62dd2302a9a35cf43b4e5e1c223e8e2f: Status 404 returned error can't find the container with id 095acec6f0383730e36a5541ca1f881b62dd2302a9a35cf43b4e5e1c223e8e2f
Jul 18 01:23:21.527257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:21.527221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" event={"ID":"ed1e8a57-35b6-43aa-81c3-73c4e9c19859","Type":"ContainerStarted","Data":"095acec6f0383730e36a5541ca1f881b62dd2302a9a35cf43b4e5e1c223e8e2f"}
Jul 18 01:23:22.534770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:22.534666    2567 generic.go:356] "Generic (PLEG): container finished" podID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerID="1cf7a8a75919d1e8433a91b1a26c3997c7ad2f1b3e9748eeb7080dedd324c8bc" exitCode=6
Jul 18 01:23:22.534770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:22.534757    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" event={"ID":"ed1e8a57-35b6-43aa-81c3-73c4e9c19859","Type":"ContainerDied","Data":"1cf7a8a75919d1e8433a91b1a26c3997c7ad2f1b3e9748eeb7080dedd324c8bc"}
Jul 18 01:23:22.535316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:22.534958    2567 scope.go:117] "RemoveContainer" containerID="1cf7a8a75919d1e8433a91b1a26c3997c7ad2f1b3e9748eeb7080dedd324c8bc"
Jul 18 01:23:22.913443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:22.913408    2567 scope.go:117] "RemoveContainer" containerID="1c1c7c957b73407cf457ace62839621e7c5d9ebbeed7aedd18fa4505f8efa217"
Jul 18 01:23:23.542132 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:23.542037    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerStarted","Data":"fc300caaf14a334f542d61287d811cfc215877078cf1178d324362a90faaae84"}
Jul 18 01:23:23.543783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:23.543756    2567 generic.go:356] "Generic (PLEG): container finished" podID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerID="fe2f1b36b27c024da4b2b2850fab3176d54fa26b544fc8588c9107eb96f2b171" exitCode=6
Jul 18 01:23:23.543911 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:23.543885    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" event={"ID":"ed1e8a57-35b6-43aa-81c3-73c4e9c19859","Type":"ContainerDied","Data":"fe2f1b36b27c024da4b2b2850fab3176d54fa26b544fc8588c9107eb96f2b171"}
Jul 18 01:23:23.543957 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:23.543947    2567 scope.go:117] "RemoveContainer" containerID="1cf7a8a75919d1e8433a91b1a26c3997c7ad2f1b3e9748eeb7080dedd324c8bc"
Jul 18 01:23:23.544010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:23.543988    2567 scope.go:117] "RemoveContainer" containerID="fe2f1b36b27c024da4b2b2850fab3176d54fa26b544fc8588c9107eb96f2b171"
Jul 18 01:23:23.544195 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:23.544177    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4_odh-ai-gateway-infra(ed1e8a57-35b6-43aa-81c3-73c4e9c19859)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859"
Jul 18 01:23:23.913324 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:23.913253    2567 scope.go:117] "RemoveContainer" containerID="7170a1f94325a88d71aaba33f00def795f3dc57fce8a058b40601dbf54c416c9"
Jul 18 01:23:24.550816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.550724    2567 generic.go:356] "Generic (PLEG): container finished" podID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerID="fc300caaf14a334f542d61287d811cfc215877078cf1178d324362a90faaae84" exitCode=6
Jul 18 01:23:24.550816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.550792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerDied","Data":"fc300caaf14a334f542d61287d811cfc215877078cf1178d324362a90faaae84"}
Jul 18 01:23:24.551260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.550853    2567 scope.go:117] "RemoveContainer" containerID="1c1c7c957b73407cf457ace62839621e7c5d9ebbeed7aedd18fa4505f8efa217"
Jul 18 01:23:24.551260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.551123    2567 scope.go:117] "RemoveContainer" containerID="fc300caaf14a334f542d61287d811cfc215877078cf1178d324362a90faaae84"
Jul 18 01:23:24.551419 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:24.551396    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-whbbw_odh-ai-gateway-infra(fd662b46-608b-471f-929c-ec78f3f58cfe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe"
Jul 18 01:23:24.553226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.553203    2567 scope.go:117] "RemoveContainer" containerID="fe2f1b36b27c024da4b2b2850fab3176d54fa26b544fc8588c9107eb96f2b171"
Jul 18 01:23:24.553451 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:24.553429    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4_odh-ai-gateway-infra(ed1e8a57-35b6-43aa-81c3-73c4e9c19859)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859"
Jul 18 01:23:24.555142 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.555116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" event={"ID":"58a8dc52-957f-46da-83f1-3e523cd08a7b","Type":"ContainerStarted","Data":"71de57a84906006374cc627bfda3ef135f8c014e2b7cb514e3f04f89a0b741d0"}
Jul 18 01:23:24.566397 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.566377    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"]
Jul 18 01:23:24.588011 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:24.587962    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" podStartSLOduration=16.587947072 podStartE2EDuration="16.587947072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:23:24.587043132 +0000 UTC m=+2584.114548091" watchObservedRunningTime="2026-07-18 01:23:24.587947072 +0000 UTC m=+2584.115452031"
Jul 18 01:23:25.535648 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.535610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:25.564685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.564646    2567 generic.go:356] "Generic (PLEG): container finished" podID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerID="71de57a84906006374cc627bfda3ef135f8c014e2b7cb514e3f04f89a0b741d0" exitCode=6
Jul 18 01:23:25.565190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.564698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" event={"ID":"58a8dc52-957f-46da-83f1-3e523cd08a7b","Type":"ContainerDied","Data":"71de57a84906006374cc627bfda3ef135f8c014e2b7cb514e3f04f89a0b741d0"}
Jul 18 01:23:25.565190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.564767    2567 scope.go:117] "RemoveContainer" containerID="7170a1f94325a88d71aaba33f00def795f3dc57fce8a058b40601dbf54c416c9"
Jul 18 01:23:25.565190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.565108    2567 scope.go:117] "RemoveContainer" containerID="71de57a84906006374cc627bfda3ef135f8c014e2b7cb514e3f04f89a0b741d0"
Jul 18 01:23:25.565435 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:25.565393    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z_odh-ai-gateway-infra(58a8dc52-957f-46da-83f1-3e523cd08a7b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b"
Jul 18 01:23:25.596956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.596920    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"]
Jul 18 01:23:25.615052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.615026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:25.707149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.707117    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:25.847659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.847630    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd662b46-608b-471f-929c-ec78f3f58cfe-maas-api-service-ca\") pod \"fd662b46-608b-471f-929c-ec78f3f58cfe\" (UID: \"fd662b46-608b-471f-929c-ec78f3f58cfe\") "
Jul 18 01:23:25.848039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.848017    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd662b46-608b-471f-929c-ec78f3f58cfe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd662b46-608b-471f-929c-ec78f3f58cfe" (UID: "fd662b46-608b-471f-929c-ec78f3f58cfe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:23:25.948618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:25.948585    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd662b46-608b-471f-929c-ec78f3f58cfe-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:23:26.227886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.227803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:26.571625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.571594    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"
Jul 18 01:23:26.572118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.571630    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw" event={"ID":"fd662b46-608b-471f-929c-ec78f3f58cfe","Type":"ContainerDied","Data":"3048b6ced9b8fccb27ce8b8660f73e68cf303e649daf1cd518faf2011a73fc98"}
Jul 18 01:23:26.572118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.571701    2567 scope.go:117] "RemoveContainer" containerID="fc300caaf14a334f542d61287d811cfc215877078cf1178d324362a90faaae84"
Jul 18 01:23:26.602482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.602438    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"]
Jul 18 01:23:26.608623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.608597    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-whbbw"]
Jul 18 01:23:26.714931 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.714906    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:26.857110 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.857028    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a8dc52-957f-46da-83f1-3e523cd08a7b-maas-api-service-ca\") pod \"58a8dc52-957f-46da-83f1-3e523cd08a7b\" (UID: \"58a8dc52-957f-46da-83f1-3e523cd08a7b\") "
Jul 18 01:23:26.857456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.857434    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a8dc52-957f-46da-83f1-3e523cd08a7b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "58a8dc52-957f-46da-83f1-3e523cd08a7b" (UID: "58a8dc52-957f-46da-83f1-3e523cd08a7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:23:26.919619 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.919585    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" path="/var/lib/kubelet/pods/fd662b46-608b-471f-929c-ec78f3f58cfe/volumes"
Jul 18 01:23:26.958014 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:26.957979    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a8dc52-957f-46da-83f1-3e523cd08a7b-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:23:27.580459 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:27.580428    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"
Jul 18 01:23:27.580895 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:27.580420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z" event={"ID":"58a8dc52-957f-46da-83f1-3e523cd08a7b","Type":"ContainerDied","Data":"63abc14923e17914c6623ff90510b5f235e35851f3a8c7d069c3429a185bcd49"}
Jul 18 01:23:27.580895 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:27.580565    2567 scope.go:117] "RemoveContainer" containerID="71de57a84906006374cc627bfda3ef135f8c014e2b7cb514e3f04f89a0b741d0"
Jul 18 01:23:27.596306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:27.596261    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"]
Jul 18 01:23:27.602294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:27.602246    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-zpf2z"]
Jul 18 01:23:28.921284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:28.921234    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" path="/var/lib/kubelet/pods/58a8dc52-957f-46da-83f1-3e523cd08a7b/volumes"
Jul 18 01:23:30.057155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:30.056653    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:23:30.830087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:30.830053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:31.913734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:31.913703    2567 scope.go:117] "RemoveContainer" containerID="84bca1ce741250c8616e630ed0d2f7424b3900f59cacd44458dcac4ddef76fda"
Jul 18 01:23:32.237022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.236937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:32.347177 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.347145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:32.609907 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.609876    2567 generic.go:356] "Generic (PLEG): container finished" podID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerID="0c846c0ce5794d3bba1364b504e1215c30f3ca4c13a8af89ac91d83976b868f2" exitCode=6
Jul 18 01:23:32.610090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.609946    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" event={"ID":"6fab7f27-4a69-422b-a46b-80f1781beb13","Type":"ContainerDied","Data":"0c846c0ce5794d3bba1364b504e1215c30f3ca4c13a8af89ac91d83976b868f2"}
Jul 18 01:23:32.610090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.610010    2567 scope.go:117] "RemoveContainer" containerID="84bca1ce741250c8616e630ed0d2f7424b3900f59cacd44458dcac4ddef76fda"
Jul 18 01:23:32.610399 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.610380    2567 scope.go:117] "RemoveContainer" containerID="0c846c0ce5794d3bba1364b504e1215c30f3ca4c13a8af89ac91d83976b868f2"
Jul 18 01:23:32.610614 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:32.610595    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-z2685_odh-ai-gateway-infra(6fab7f27-4a69-422b-a46b-80f1781beb13)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13"
Jul 18 01:23:32.663871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.663836    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"]
Jul 18 01:23:32.664479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664455    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:32.664562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664486    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:32.664562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664516    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:32.664562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664524    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:32.664562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664549    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.664562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664559    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664569    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664577    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664679    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664691    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664700    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664709    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:32.664827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.664719    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:32.669521 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.669497    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.672579 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.672553    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8d0c59be-openshift-default-dockercfg-tmsxm\""
Jul 18 01:23:32.696964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.696932    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"]
Jul 18 01:23:32.714828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.714798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:32.812803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.812766    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-podinfo\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.812974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.812811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8px7x\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-kube-api-access-8px7x\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.812974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.812869    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-data\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.812974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.812905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-socket\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.812974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.812938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d630c29b-06df-4cff-8b2e-3844c4181f0c-istiod-ca-cert\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.813115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.813012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-envoy\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.813115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.813048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-credential-socket\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.813115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.813096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-certs\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.813208 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.813115    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-token\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.913624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-token\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.913624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913630    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-podinfo\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8px7x\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-kube-api-access-8px7x\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-data\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-socket\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d630c29b-06df-4cff-8b2e-3844c4181f0c-istiod-ca-cert\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-envoy\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-credential-socket\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.913959    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-certs\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.914178    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-socket\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.914359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-certs\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.914382    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-credential-socket\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.914497    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-data\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.914813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.914789    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d630c29b-06df-4cff-8b2e-3844c4181f0c-istiod-ca-cert\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.916394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.916366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-podinfo\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.916496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.916395    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-envoy\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.923015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.922984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-token\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.923852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.923835    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8px7x\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-kube-api-access-8px7x\") pod \"e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") " pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:32.980936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:32.980896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:33.120742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.120709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:33.135776 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.135750    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"]
Jul 18 01:23:33.137532 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:33.137507    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd630c29b_06df_4cff_8b2e_3844c4181f0c.slice/crio-a1201df0ffc741477135f77a9d72f094cd495faf32f43cfee3029770c141e3d8 WatchSource:0}: Error finding container a1201df0ffc741477135f77a9d72f094cd495faf32f43cfee3029770c141e3d8: Status 404 returned error can't find the container with id a1201df0ffc741477135f77a9d72f094cd495faf32f43cfee3029770c141e3d8
Jul 18 01:23:33.139760 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.139726    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:23:33.139843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.139793    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:23:33.139843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.139821    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:23:33.617874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.617832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" event={"ID":"d630c29b-06df-4cff-8b2e-3844c4181f0c","Type":"ContainerStarted","Data":"fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"}
Jul 18 01:23:33.618083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.617894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" event={"ID":"d630c29b-06df-4cff-8b2e-3844c4181f0c","Type":"ContainerStarted","Data":"a1201df0ffc741477135f77a9d72f094cd495faf32f43cfee3029770c141e3d8"}
Jul 18 01:23:33.636443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.636415    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"]
Jul 18 01:23:33.638873 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.638821    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" podStartSLOduration=1.63880632 podStartE2EDuration="1.63880632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:23:33.637595077 +0000 UTC m=+2593.165100037" watchObservedRunningTime="2026-07-18 01:23:33.63880632 +0000 UTC m=+2593.166311281"
Jul 18 01:23:33.778819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.778793    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:33.925028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.924936    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fab7f27-4a69-422b-a46b-80f1781beb13-maas-api-service-ca\") pod \"6fab7f27-4a69-422b-a46b-80f1781beb13\" (UID: \"6fab7f27-4a69-422b-a46b-80f1781beb13\") "
Jul 18 01:23:33.925473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.925262    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fab7f27-4a69-422b-a46b-80f1781beb13-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6fab7f27-4a69-422b-a46b-80f1781beb13" (UID: "6fab7f27-4a69-422b-a46b-80f1781beb13"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:23:33.981033 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:33.980994    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:34.026562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.026525    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fab7f27-4a69-422b-a46b-80f1781beb13-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:23:34.625958 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.625929    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"
Jul 18 01:23:34.626113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.625960    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685" event={"ID":"6fab7f27-4a69-422b-a46b-80f1781beb13","Type":"ContainerDied","Data":"68b9f76b4baadfd11775ebb105143bde8708a937b1de1f89b78198581e49d576"}
Jul 18 01:23:34.626113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.626018    2567 scope.go:117] "RemoveContainer" containerID="0c846c0ce5794d3bba1364b504e1215c30f3ca4c13a8af89ac91d83976b868f2"
Jul 18 01:23:34.649856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.649825    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"]
Jul 18 01:23:34.656975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.656947    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-z2685"]
Jul 18 01:23:34.918967 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.918883    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" path="/var/lib/kubelet/pods/6fab7f27-4a69-422b-a46b-80f1781beb13/volumes"
Jul 18 01:23:34.987234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:34.987205    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:35.569132 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.569095    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:35.632153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.632117    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:35.633158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.633136    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:35.660780 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.660754    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:35.726399 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.726364    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:35.845854 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.845754    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:35.913047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:35.913020    2567 scope.go:117] "RemoveContainer" containerID="fe2f1b36b27c024da4b2b2850fab3176d54fa26b544fc8588c9107eb96f2b171"
Jul 18 01:23:36.281341 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:36.281308    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:36.382350 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:36.382320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:36.639662 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:36.639627    2567 generic.go:356] "Generic (PLEG): container finished" podID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerID="ad76b696f6754b18395955dfa24302d8cb24437310926b35fa497e1e9e0bfda0" exitCode=6
Jul 18 01:23:36.639855 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:36.639695    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" event={"ID":"ed1e8a57-35b6-43aa-81c3-73c4e9c19859","Type":"ContainerDied","Data":"ad76b696f6754b18395955dfa24302d8cb24437310926b35fa497e1e9e0bfda0"}
Jul 18 01:23:36.639855 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:36.639759    2567 scope.go:117] "RemoveContainer" containerID="fe2f1b36b27c024da4b2b2850fab3176d54fa26b544fc8588c9107eb96f2b171"
Jul 18 01:23:36.640128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:36.640103    2567 scope.go:117] "RemoveContainer" containerID="ad76b696f6754b18395955dfa24302d8cb24437310926b35fa497e1e9e0bfda0"
Jul 18 01:23:36.640386 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:23:36.640358    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4_odh-ai-gateway-infra(ed1e8a57-35b6-43aa-81c3-73c4e9c19859)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859"
Jul 18 01:23:37.665904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:37.665868    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"]
Jul 18 01:23:37.812511 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:37.812484    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:37.863285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:37.863240    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e8a57-35b6-43aa-81c3-73c4e9c19859-maas-api-service-ca\") pod \"ed1e8a57-35b6-43aa-81c3-73c4e9c19859\" (UID: \"ed1e8a57-35b6-43aa-81c3-73c4e9c19859\") "
Jul 18 01:23:37.863647 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:37.863617    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed1e8a57-35b6-43aa-81c3-73c4e9c19859-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed1e8a57-35b6-43aa-81c3-73c4e9c19859" (UID: "ed1e8a57-35b6-43aa-81c3-73c4e9c19859"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:23:37.964306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:37.964165    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e8a57-35b6-43aa-81c3-73c4e9c19859-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:23:38.653703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:38.653667    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4" event={"ID":"ed1e8a57-35b6-43aa-81c3-73c4e9c19859","Type":"ContainerDied","Data":"095acec6f0383730e36a5541ca1f881b62dd2302a9a35cf43b4e5e1c223e8e2f"}
Jul 18 01:23:38.653703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:38.653690    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"
Jul 18 01:23:38.653987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:38.653736    2567 scope.go:117] "RemoveContainer" containerID="ad76b696f6754b18395955dfa24302d8cb24437310926b35fa497e1e9e0bfda0"
Jul 18 01:23:38.676721 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:38.676693    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"]
Jul 18 01:23:38.682777 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:38.682753    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-b6rg4"]
Jul 18 01:23:38.918762 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:38.918662    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" path="/var/lib/kubelet/pods/ed1e8a57-35b6-43aa-81c3-73c4e9c19859/volumes"
Jul 18 01:23:39.207669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:39.207584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:23:40.032217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:40.032185    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:40.716003 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:40.715971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:40.813435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:40.813403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:41.136374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:41.136303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:46.858368 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:46.858335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:46.958554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:46.958507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:47.219906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.219805    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"]
Jul 18 01:23:47.220200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220186    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220202    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220212    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220217    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220230    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220236    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220242    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:47.220244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220247    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd662b46-608b-471f-929c-ec78f3f58cfe" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220257    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220261    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220285    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220290    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220350    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220358    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220363    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220370    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a8dc52-957f-46da-83f1-3e523cd08a7b" containerName="revoke-keys"
Jul 18 01:23:47.220481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.220376    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:47.223372 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.223354    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:47.226601 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.226584    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:23:47.226601 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.226594    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:23:47.232850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.232825    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"]
Jul 18 01:23:47.235527 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.235500    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7589b4a-d1ad-474f-b053-f350da25f0c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt\" (UID: \"a7589b4a-d1ad-474f-b053-f350da25f0c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:47.336880 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.336847    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7589b4a-d1ad-474f-b053-f350da25f0c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt\" (UID: \"a7589b4a-d1ad-474f-b053-f350da25f0c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:47.337500 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.337479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7589b4a-d1ad-474f-b053-f350da25f0c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt\" (UID: \"a7589b4a-d1ad-474f-b053-f350da25f0c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:47.534689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.534600    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:47.678177 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.678146    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"]
Jul 18 01:23:47.680416 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:47.680389    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7589b4a_d1ad_474f_b053_f350da25f0c9.slice/crio-fe77d567de0bb530185d8c82e3b24fc29facda48d88b005fa7e096a73a8347c8 WatchSource:0}: Error finding container fe77d567de0bb530185d8c82e3b24fc29facda48d88b005fa7e096a73a8347c8: Status 404 returned error can't find the container with id fe77d567de0bb530185d8c82e3b24fc29facda48d88b005fa7e096a73a8347c8
Jul 18 01:23:47.719863 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.719829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt" event={"ID":"a7589b4a-d1ad-474f-b053-f350da25f0c9","Type":"ContainerStarted","Data":"fe77d567de0bb530185d8c82e3b24fc29facda48d88b005fa7e096a73a8347c8"}
Jul 18 01:23:47.721669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.721645    2567 generic.go:356] "Generic (PLEG): container finished" podID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerID="fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a" exitCode=0
Jul 18 01:23:47.721795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.721722    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" event={"ID":"d630c29b-06df-4cff-8b2e-3844c4181f0c","Type":"ContainerDied","Data":"fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"}
Jul 18 01:23:47.722151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.722134    2567 scope.go:117] "RemoveContainer" containerID="fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"
Jul 18 01:23:47.723186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.723150    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:23:47.723314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.723225    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:23:47.723314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:47.723287    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:23:48.728083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:48.727982    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7589b4a-d1ad-474f-b053-f350da25f0c9" containerID="cff819ffeced75ffc5c935487f476900bd118970ae217a1926b4c06bbbc1f3e0" exitCode=0
Jul 18 01:23:48.728083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:48.728069    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt" event={"ID":"a7589b4a-d1ad-474f-b053-f350da25f0c9","Type":"ContainerDied","Data":"cff819ffeced75ffc5c935487f476900bd118970ae217a1926b4c06bbbc1f3e0"}
Jul 18 01:23:48.730046 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:48.730019    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" event={"ID":"d630c29b-06df-4cff-8b2e-3844c4181f0c","Type":"ContainerStarted","Data":"032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1"}
Jul 18 01:23:48.981699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:48.981592    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:48.982950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:48.982921    2567 patch_prober.go:28] interesting pod/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 18 01:23:48.983072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:48.982982    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 18 01:23:49.907390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:49.907365    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:49.976986 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:49.976947    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7589b4a-d1ad-474f-b053-f350da25f0c9-maas-api-service-ca\") pod \"a7589b4a-d1ad-474f-b053-f350da25f0c9\" (UID: \"a7589b4a-d1ad-474f-b053-f350da25f0c9\") "
Jul 18 01:23:49.977384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:49.977358    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7589b4a-d1ad-474f-b053-f350da25f0c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7589b4a-d1ad-474f-b053-f350da25f0c9" (UID: "a7589b4a-d1ad-474f-b053-f350da25f0c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:23:49.982196 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:49.982170    2567 patch_prober.go:28] interesting pod/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 18 01:23:49.982342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:49.982223    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 18 01:23:50.078494 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.078453    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7589b4a-d1ad-474f-b053-f350da25f0c9-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:23:50.632513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.632476    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:50.742205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.742179    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt"
Jul 18 01:23:50.742392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.742221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt" event={"ID":"a7589b4a-d1ad-474f-b053-f350da25f0c9","Type":"ContainerDied","Data":"fe77d567de0bb530185d8c82e3b24fc29facda48d88b005fa7e096a73a8347c8"}
Jul 18 01:23:50.742392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.742262    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe77d567de0bb530185d8c82e3b24fc29facda48d88b005fa7e096a73a8347c8"
Jul 18 01:23:50.981419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.981332    2567 patch_prober.go:28] interesting pod/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 18 01:23:50.981419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:50.981385    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 18 01:23:51.981519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:51.981466    2567 patch_prober.go:28] interesting pod/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 18 01:23:51.981955 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:51.981522    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 18 01:23:52.985403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:52.985373    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:52.986517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:52.986497    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:23:57.705968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:57.705924    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:23:58.902501 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902471    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"]
Jul 18 01:23:58.902886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902829    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7589b4a-d1ad-474f-b053-f350da25f0c9" containerName="revoke-keys"
Jul 18 01:23:58.902886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902840    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7589b4a-d1ad-474f-b053-f350da25f0c9" containerName="revoke-keys"
Jul 18 01:23:58.902886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902853    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:58.902886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902860    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:58.902886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902873    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:58.902886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902879    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:58.903072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902933    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7589b4a-d1ad-474f-b053-f350da25f0c9" containerName="revoke-keys"
Jul 18 01:23:58.903072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902940    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1e8a57-35b6-43aa-81c3-73c4e9c19859" containerName="revoke-keys"
Jul 18 01:23:58.903072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.902949    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fab7f27-4a69-422b-a46b-80f1781beb13" containerName="revoke-keys"
Jul 18 01:23:58.905987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.905966    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:23:58.908325 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.908306    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:23:58.908441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.908314    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:23:58.912351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.912329    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"]
Jul 18 01:23:58.958091 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:58.958055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh\" (UID: \"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:23:59.058612 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.058582    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh\" (UID: \"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:23:59.059256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.059229    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh\" (UID: \"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:23:59.216933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.216832    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:23:59.356331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.356300    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"]
Jul 18 01:23:59.358367 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:23:59.358322    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc874c0d8_ddc7_41b7_b4e6_60dbd497ce2c.slice/crio-016bbd5eb529656f96a07a1fd0be4abd8bd67fceb29029d94fd242bbaa97bf38 WatchSource:0}: Error finding container 016bbd5eb529656f96a07a1fd0be4abd8bd67fceb29029d94fd242bbaa97bf38: Status 404 returned error can't find the container with id 016bbd5eb529656f96a07a1fd0be4abd8bd67fceb29029d94fd242bbaa97bf38
Jul 18 01:23:59.792333 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.792295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" event={"ID":"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c","Type":"ContainerStarted","Data":"3cf991ef33e22c512ff829287e327307a5daeb9f73b7dc8779fac9e1c0d902a4"}
Jul 18 01:23:59.792529 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.792364    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" event={"ID":"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c","Type":"ContainerStarted","Data":"016bbd5eb529656f96a07a1fd0be4abd8bd67fceb29029d94fd242bbaa97bf38"}
Jul 18 01:23:59.808144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:23:59.807974    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" podStartSLOduration=1.807954521 podStartE2EDuration="1.807954521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:23:59.80783073 +0000 UTC m=+2619.335335687" watchObservedRunningTime="2026-07-18 01:23:59.807954521 +0000 UTC m=+2619.335459482"
Jul 18 01:24:00.616516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:00.616476    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:24:00.798424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:00.798394    2567 generic.go:356] "Generic (PLEG): container finished" podID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerID="3cf991ef33e22c512ff829287e327307a5daeb9f73b7dc8779fac9e1c0d902a4" exitCode=6
Jul 18 01:24:00.798615 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:00.798484    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" event={"ID":"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c","Type":"ContainerDied","Data":"3cf991ef33e22c512ff829287e327307a5daeb9f73b7dc8779fac9e1c0d902a4"}
Jul 18 01:24:00.798682 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:00.798669    2567 scope.go:117] "RemoveContainer" containerID="3cf991ef33e22c512ff829287e327307a5daeb9f73b7dc8779fac9e1c0d902a4"
Jul 18 01:24:01.805236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:01.805198    2567 generic.go:356] "Generic (PLEG): container finished" podID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerID="3a60515768f8519b4ef674dca4524b7112b1db1ed073c4a9f9ddd3be0479a299" exitCode=6
Jul 18 01:24:01.805833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:01.805300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" event={"ID":"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c","Type":"ContainerDied","Data":"3a60515768f8519b4ef674dca4524b7112b1db1ed073c4a9f9ddd3be0479a299"}
Jul 18 01:24:01.805833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:01.805365    2567 scope.go:117] "RemoveContainer" containerID="3cf991ef33e22c512ff829287e327307a5daeb9f73b7dc8779fac9e1c0d902a4"
Jul 18 01:24:01.805833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:01.805650    2567 scope.go:117] "RemoveContainer" containerID="3a60515768f8519b4ef674dca4524b7112b1db1ed073c4a9f9ddd3be0479a299"
Jul 18 01:24:01.805995 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:01.805866    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh_odh-ai-gateway-infra(c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c"
Jul 18 01:24:02.308623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:02.308592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:02.812592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:02.812566    2567 scope.go:117] "RemoveContainer" containerID="3a60515768f8519b4ef674dca4524b7112b1db1ed073c4a9f9ddd3be0479a299"
Jul 18 01:24:02.812968 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:02.812832    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh_odh-ai-gateway-infra(c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c"
Jul 18 01:24:03.509192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.509154    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"]
Jul 18 01:24:03.514056 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.514034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:03.519744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.519716    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"]
Jul 18 01:24:03.596160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.596121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ebb6e26-92d4-4623-971c-560c5d68e756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-6h76g\" (UID: \"8ebb6e26-92d4-4623-971c-560c5d68e756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:03.696995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.696941    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ebb6e26-92d4-4623-971c-560c5d68e756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-6h76g\" (UID: \"8ebb6e26-92d4-4623-971c-560c5d68e756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:03.697666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.697641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ebb6e26-92d4-4623-971c-560c5d68e756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-6h76g\" (UID: \"8ebb6e26-92d4-4623-971c-560c5d68e756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:03.826351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.826319    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:03.975030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:03.974942    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"]
Jul 18 01:24:03.977318 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:24:03.977295    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ebb6e26_92d4_4623_971c_560c5d68e756.slice/crio-bcdc7b1f47c0ce09cac1ef5ad381d5d75bbd5dbeba4684b3f93bd126ada86dbd WatchSource:0}: Error finding container bcdc7b1f47c0ce09cac1ef5ad381d5d75bbd5dbeba4684b3f93bd126ada86dbd: Status 404 returned error can't find the container with id bcdc7b1f47c0ce09cac1ef5ad381d5d75bbd5dbeba4684b3f93bd126ada86dbd
Jul 18 01:24:04.827914 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:04.827880    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerID="27d44a1038e2d5ae947e625045af7ae74e7271c10fb708c3a268cbcc7aa033fb" exitCode=6
Jul 18 01:24:04.828370 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:04.827966    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" event={"ID":"8ebb6e26-92d4-4623-971c-560c5d68e756","Type":"ContainerDied","Data":"27d44a1038e2d5ae947e625045af7ae74e7271c10fb708c3a268cbcc7aa033fb"}
Jul 18 01:24:04.828370 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:04.828018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" event={"ID":"8ebb6e26-92d4-4623-971c-560c5d68e756","Type":"ContainerStarted","Data":"bcdc7b1f47c0ce09cac1ef5ad381d5d75bbd5dbeba4684b3f93bd126ada86dbd"}
Jul 18 01:24:04.828370 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:04.828146    2567 scope.go:117] "RemoveContainer" containerID="27d44a1038e2d5ae947e625045af7ae74e7271c10fb708c3a268cbcc7aa033fb"
Jul 18 01:24:05.834993 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:05.834957    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerID="dee53103552e17b1c88ba25837713b1bbc2c225dd53021c6c890b1ecf4a1808e" exitCode=6
Jul 18 01:24:05.835447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:05.835019    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" event={"ID":"8ebb6e26-92d4-4623-971c-560c5d68e756","Type":"ContainerDied","Data":"dee53103552e17b1c88ba25837713b1bbc2c225dd53021c6c890b1ecf4a1808e"}
Jul 18 01:24:05.835447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:05.835095    2567 scope.go:117] "RemoveContainer" containerID="27d44a1038e2d5ae947e625045af7ae74e7271c10fb708c3a268cbcc7aa033fb"
Jul 18 01:24:05.835447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:05.835338    2567 scope.go:117] "RemoveContainer" containerID="dee53103552e17b1c88ba25837713b1bbc2c225dd53021c6c890b1ecf4a1808e"
Jul 18 01:24:05.835611 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:05.835588    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-6h76g_odh-ai-gateway-infra(8ebb6e26-92d4-4623-971c-560c5d68e756)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756"
Jul 18 01:24:06.112326 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.112208    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"]
Jul 18 01:24:06.116849 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.116832    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:06.123148 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.122975    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"]
Jul 18 01:24:06.223687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.223644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64980ad9-d542-47fd-ab0a-019291805fc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w\" (UID: \"64980ad9-d542-47fd-ab0a-019291805fc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:06.324976 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.324941    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64980ad9-d542-47fd-ab0a-019291805fc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w\" (UID: \"64980ad9-d542-47fd-ab0a-019291805fc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:06.325623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.325605    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64980ad9-d542-47fd-ab0a-019291805fc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w\" (UID: \"64980ad9-d542-47fd-ab0a-019291805fc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:06.430158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.430066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:06.570870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.570847    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"]
Jul 18 01:24:06.572795 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:24:06.572762    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64980ad9_d542_47fd_ab0a_019291805fc3.slice/crio-4eefb652ee64e5b107c9e8bc61200ce66486608bb613a8007bc6e149cd394a3c WatchSource:0}: Error finding container 4eefb652ee64e5b107c9e8bc61200ce66486608bb613a8007bc6e149cd394a3c: Status 404 returned error can't find the container with id 4eefb652ee64e5b107c9e8bc61200ce66486608bb613a8007bc6e149cd394a3c
Jul 18 01:24:06.841171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.841138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" event={"ID":"64980ad9-d542-47fd-ab0a-019291805fc3","Type":"ContainerStarted","Data":"4eefb652ee64e5b107c9e8bc61200ce66486608bb613a8007bc6e149cd394a3c"}
Jul 18 01:24:06.842885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:06.842865    2567 scope.go:117] "RemoveContainer" containerID="dee53103552e17b1c88ba25837713b1bbc2c225dd53021c6c890b1ecf4a1808e"
Jul 18 01:24:06.843076 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:06.843058    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-6h76g_odh-ai-gateway-infra(8ebb6e26-92d4-4623-971c-560c5d68e756)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756"
Jul 18 01:24:07.849170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:07.849136    2567 generic.go:356] "Generic (PLEG): container finished" podID="64980ad9-d542-47fd-ab0a-019291805fc3" containerID="4763165a63b762c185ceb62e7d9adc3ec09920d3e074b9ef8a1b4c9923da1afc" exitCode=6
Jul 18 01:24:07.849633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:07.849224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" event={"ID":"64980ad9-d542-47fd-ab0a-019291805fc3","Type":"ContainerDied","Data":"4763165a63b762c185ceb62e7d9adc3ec09920d3e074b9ef8a1b4c9923da1afc"}
Jul 18 01:24:07.849633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:07.849428    2567 scope.go:117] "RemoveContainer" containerID="4763165a63b762c185ceb62e7d9adc3ec09920d3e074b9ef8a1b4c9923da1afc"
Jul 18 01:24:07.859124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:07.859094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:08.000533 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:08.000501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:08.855758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:08.855725    2567 generic.go:356] "Generic (PLEG): container finished" podID="64980ad9-d542-47fd-ab0a-019291805fc3" containerID="c864a1c29e5a056058b488225fadf1d5edbd45b15222a3ce9a2d55d230029134" exitCode=6
Jul 18 01:24:08.856253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:08.855797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" event={"ID":"64980ad9-d542-47fd-ab0a-019291805fc3","Type":"ContainerDied","Data":"c864a1c29e5a056058b488225fadf1d5edbd45b15222a3ce9a2d55d230029134"}
Jul 18 01:24:08.856253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:08.855848    2567 scope.go:117] "RemoveContainer" containerID="4763165a63b762c185ceb62e7d9adc3ec09920d3e074b9ef8a1b4c9923da1afc"
Jul 18 01:24:08.856253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:08.856084    2567 scope.go:117] "RemoveContainer" containerID="c864a1c29e5a056058b488225fadf1d5edbd45b15222a3ce9a2d55d230029134"
Jul 18 01:24:08.856467 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:08.856366    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w_odh-ai-gateway-infra(64980ad9-d542-47fd-ab0a-019291805fc3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" podUID="64980ad9-d542-47fd-ab0a-019291805fc3"
Jul 18 01:24:09.862944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:09.862918    2567 scope.go:117] "RemoveContainer" containerID="c864a1c29e5a056058b488225fadf1d5edbd45b15222a3ce9a2d55d230029134"
Jul 18 01:24:09.863431 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:09.863183    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w_odh-ai-gateway-infra(64980ad9-d542-47fd-ab0a-019291805fc3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" podUID="64980ad9-d542-47fd-ab0a-019291805fc3"
Jul 18 01:24:11.050352 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.050315    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"]
Jul 18 01:24:11.055096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.055072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:11.061510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.061483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"]
Jul 18 01:24:11.172037 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.172005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq\" (UID: \"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:11.273565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.273523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq\" (UID: \"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:11.274240 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.274212    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq\" (UID: \"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:11.368197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.368160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:11.510262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.510225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"]
Jul 18 01:24:11.513336 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:24:11.513307    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12cc9d2a_e2ba_4abf_a5f7_e73300645d6e.slice/crio-7c6403d9a8a7d4ba407955c9b1c60abdbd2a2c0f23ac4c10f6903ab96a92220a WatchSource:0}: Error finding container 7c6403d9a8a7d4ba407955c9b1c60abdbd2a2c0f23ac4c10f6903ab96a92220a: Status 404 returned error can't find the container with id 7c6403d9a8a7d4ba407955c9b1c60abdbd2a2c0f23ac4c10f6903ab96a92220a
Jul 18 01:24:11.876306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.876254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" event={"ID":"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e","Type":"ContainerStarted","Data":"32e8cb93f5af4e28f9dfe82682480b71eb523ce756986036f27048f8222bab92"}
Jul 18 01:24:11.876451 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.876337    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" event={"ID":"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e","Type":"ContainerStarted","Data":"7c6403d9a8a7d4ba407955c9b1c60abdbd2a2c0f23ac4c10f6903ab96a92220a"}
Jul 18 01:24:11.893032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:11.892979    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" podStartSLOduration=0.892958041 podStartE2EDuration="892.958041ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:24:11.892282791 +0000 UTC m=+2631.419787752" watchObservedRunningTime="2026-07-18 01:24:11.892958041 +0000 UTC m=+2631.420463002"
Jul 18 01:24:12.883692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:12.883652    2567 generic.go:356] "Generic (PLEG): container finished" podID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerID="32e8cb93f5af4e28f9dfe82682480b71eb523ce756986036f27048f8222bab92" exitCode=6
Jul 18 01:24:12.884098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:12.883741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" event={"ID":"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e","Type":"ContainerDied","Data":"32e8cb93f5af4e28f9dfe82682480b71eb523ce756986036f27048f8222bab92"}
Jul 18 01:24:12.884098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:12.883878    2567 scope.go:117] "RemoveContainer" containerID="32e8cb93f5af4e28f9dfe82682480b71eb523ce756986036f27048f8222bab92"
Jul 18 01:24:12.945204 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:12.945171    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"]
Jul 18 01:24:12.945443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:12.945415    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy" containerID="cri-o://032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1" gracePeriod=30
Jul 18 01:24:12.969413 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:12.969377    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:13.160313 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.160287    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:13.270374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.270344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:13.367757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.367729    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:13.481709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.481614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:13.891110 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.891069    2567 generic.go:356] "Generic (PLEG): container finished" podID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerID="649bffc9695f936b652c1939bd75b46f5224f83e90e24f2331bcc5dc20a08233" exitCode=6
Jul 18 01:24:13.891574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.891137    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" event={"ID":"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e","Type":"ContainerDied","Data":"649bffc9695f936b652c1939bd75b46f5224f83e90e24f2331bcc5dc20a08233"}
Jul 18 01:24:13.891574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.891216    2567 scope.go:117] "RemoveContainer" containerID="32e8cb93f5af4e28f9dfe82682480b71eb523ce756986036f27048f8222bab92"
Jul 18 01:24:13.891574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.891503    2567 scope.go:117] "RemoveContainer" containerID="649bffc9695f936b652c1939bd75b46f5224f83e90e24f2331bcc5dc20a08233"
Jul 18 01:24:13.891709 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:13.891691    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq_odh-ai-gateway-infra(12cc9d2a-e2ba-4abf-a5f7-e73300645d6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e"
Jul 18 01:24:13.913061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:13.913040    2567 scope.go:117] "RemoveContainer" containerID="3a60515768f8519b4ef674dca4524b7112b1db1ed073c4a9f9ddd3be0479a299"
Jul 18 01:24:14.898013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:14.897977    2567 generic.go:356] "Generic (PLEG): container finished" podID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerID="e9c2f28619d9928a24a05b8b2c7803f89760820ded0fd9949104a73264815f1b" exitCode=6
Jul 18 01:24:14.898520 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:14.898051    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" event={"ID":"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c","Type":"ContainerDied","Data":"e9c2f28619d9928a24a05b8b2c7803f89760820ded0fd9949104a73264815f1b"}
Jul 18 01:24:14.898520 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:14.898166    2567 scope.go:117] "RemoveContainer" containerID="3a60515768f8519b4ef674dca4524b7112b1db1ed073c4a9f9ddd3be0479a299"
Jul 18 01:24:14.898520 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:14.898483    2567 scope.go:117] "RemoveContainer" containerID="e9c2f28619d9928a24a05b8b2c7803f89760820ded0fd9949104a73264815f1b"
Jul 18 01:24:14.898749 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:14.898725    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh_odh-ai-gateway-infra(c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c"
Jul 18 01:24:14.899979 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:14.899963    2567 scope.go:117] "RemoveContainer" containerID="649bffc9695f936b652c1939bd75b46f5224f83e90e24f2331bcc5dc20a08233"
Jul 18 01:24:14.900142 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:14.900125    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq_odh-ai-gateway-infra(12cc9d2a-e2ba-4abf-a5f7-e73300645d6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e"
Jul 18 01:24:15.923822 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:15.923786    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"]
Jul 18 01:24:16.065145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.065121    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:24:16.223925 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.223820    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c-maas-api-service-ca\") pod \"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c\" (UID: \"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c\") "
Jul 18 01:24:16.224250 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.224222    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" (UID: "c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:24:16.324697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.324659    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:16.913631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.913597    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"
Jul 18 01:24:16.919180 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.919149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh" event={"ID":"c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c","Type":"ContainerDied","Data":"016bbd5eb529656f96a07a1fd0be4abd8bd67fceb29029d94fd242bbaa97bf38"}
Jul 18 01:24:16.919374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.919211    2567 scope.go:117] "RemoveContainer" containerID="e9c2f28619d9928a24a05b8b2c7803f89760820ded0fd9949104a73264815f1b"
Jul 18 01:24:16.967800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.967764    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"]
Jul 18 01:24:16.977570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:16.977542    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-wh9qh"]
Jul 18 01:24:18.209758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.209694    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:24:18.341366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341333    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8px7x\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-kube-api-access-8px7x\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341373    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-certs\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341628 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341420    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-socket\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341628 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341438    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-envoy\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341628 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341550    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d630c29b-06df-4cff-8b2e-3844c4181f0c-istiod-ca-cert\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341628 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341591    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-credential-socket\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341652    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-token\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341703    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-podinfo\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341736    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-data\") pod \"d630c29b-06df-4cff-8b2e-3844c4181f0c\" (UID: \"d630c29b-06df-4cff-8b2e-3844c4181f0c\") "
Jul 18 01:24:18.341845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341751    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:24:18.341845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341763    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:24:18.342101 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341914    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:24:18.342101 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.341943    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d630c29b-06df-4cff-8b2e-3844c4181f0c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:24:18.342402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.342218    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.342402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.342239    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.342402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.342253    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d630c29b-06df-4cff-8b2e-3844c4181f0c-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.342402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.342306    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.343103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.343073    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-data" (OuterVolumeSpecName: "istio-data") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:24:18.343773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.343743    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:24:18.343942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.343916    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-kube-api-access-8px7x" (OuterVolumeSpecName: "kube-api-access-8px7x") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "kube-api-access-8px7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:24:18.344034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.343937    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-token" (OuterVolumeSpecName: "istio-token") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:24:18.344075 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.344057    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d630c29b-06df-4cff-8b2e-3844c4181f0c" (UID: "d630c29b-06df-4cff-8b2e-3844c4181f0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:24:18.443704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.443654    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.443704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.443698    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.443704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.443708    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.443704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.443718    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8px7x\" (UniqueName: \"kubernetes.io/projected/d630c29b-06df-4cff-8b2e-3844c4181f0c-kube-api-access-8px7x\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.443983 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.443727    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d630c29b-06df-4cff-8b2e-3844c4181f0c-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:18.919210 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.919164    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" path="/var/lib/kubelet/pods/c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c/volumes"
Jul 18 01:24:18.925915 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.925883    2567 generic.go:356] "Generic (PLEG): container finished" podID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerID="032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1" exitCode=0
Jul 18 01:24:18.926118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.925938    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"
Jul 18 01:24:18.926118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.925965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" event={"ID":"d630c29b-06df-4cff-8b2e-3844c4181f0c","Type":"ContainerDied","Data":"032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1"}
Jul 18 01:24:18.926118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.926020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk" event={"ID":"d630c29b-06df-4cff-8b2e-3844c4181f0c","Type":"ContainerDied","Data":"a1201df0ffc741477135f77a9d72f094cd495faf32f43cfee3029770c141e3d8"}
Jul 18 01:24:18.926118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.926042    2567 scope.go:117] "RemoveContainer" containerID="032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1"
Jul 18 01:24:18.935243 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.935226    2567 scope.go:117] "RemoveContainer" containerID="fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"
Jul 18 01:24:18.942684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.942663    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"]
Jul 18 01:24:18.943767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.943750    2567 scope.go:117] "RemoveContainer" containerID="032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1"
Jul 18 01:24:18.944014 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:18.943996    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1\": container with ID starting with 032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1 not found: ID does not exist" containerID="032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1"
Jul 18 01:24:18.944084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.944021    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1"} err="failed to get container status \"032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1\": rpc error: code = NotFound desc = could not find container \"032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1\": container with ID starting with 032a369191a59e1a741ae0e38666f6cf3219c11222072b559ac2d366205347f1 not found: ID does not exist"
Jul 18 01:24:18.944084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.944039    2567 scope.go:117] "RemoveContainer" containerID="fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"
Jul 18 01:24:18.944255 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:18.944241    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a\": container with ID starting with fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a not found: ID does not exist" containerID="fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"
Jul 18 01:24:18.944361 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.944259    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a"} err="failed to get container status \"fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a\": rpc error: code = NotFound desc = could not find container \"fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a\": container with ID starting with fed7539b5df91b5f851d7587b82ecb9d2171bcb8e763fdaecf23c5f41319e85a not found: ID does not exist"
Jul 18 01:24:18.947379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:18.947358    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8d0c59be-openshift-default-9449fdc6c-xbhjk"]
Jul 18 01:24:19.177852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:19.177756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:19.237423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:19.237384    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:19.312352 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:19.312326    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:19.408011 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:19.407971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:19.512700 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:19.512612    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:20.920192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:20.920157    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" path="/var/lib/kubelet/pods/d630c29b-06df-4cff-8b2e-3844c4181f0c/volumes"
Jul 18 01:24:21.913532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:21.913505    2567 scope.go:117] "RemoveContainer" containerID="dee53103552e17b1c88ba25837713b1bbc2c225dd53021c6c890b1ecf4a1808e"
Jul 18 01:24:22.950226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:22.950189    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerID="3f62e0a8511f31283902b0e4145e10bd7e525feead705109501a42622f6b1c5d" exitCode=6
Jul 18 01:24:22.950736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:22.950249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" event={"ID":"8ebb6e26-92d4-4623-971c-560c5d68e756","Type":"ContainerDied","Data":"3f62e0a8511f31283902b0e4145e10bd7e525feead705109501a42622f6b1c5d"}
Jul 18 01:24:22.950736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:22.950315    2567 scope.go:117] "RemoveContainer" containerID="dee53103552e17b1c88ba25837713b1bbc2c225dd53021c6c890b1ecf4a1808e"
Jul 18 01:24:22.950736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:22.950645    2567 scope.go:117] "RemoveContainer" containerID="3f62e0a8511f31283902b0e4145e10bd7e525feead705109501a42622f6b1c5d"
Jul 18 01:24:22.950921 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:22.950896    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-6h76g_odh-ai-gateway-infra(8ebb6e26-92d4-4623-971c-560c5d68e756)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756"
Jul 18 01:24:23.913118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:23.913088    2567 scope.go:117] "RemoveContainer" containerID="c864a1c29e5a056058b488225fadf1d5edbd45b15222a3ce9a2d55d230029134"
Jul 18 01:24:23.979068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:23.976862    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"]
Jul 18 01:24:24.122032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.122008    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:24.295200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.295112    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ebb6e26-92d4-4623-971c-560c5d68e756-maas-api-service-ca\") pod \"8ebb6e26-92d4-4623-971c-560c5d68e756\" (UID: \"8ebb6e26-92d4-4623-971c-560c5d68e756\") "
Jul 18 01:24:24.295507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.295483    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ebb6e26-92d4-4623-971c-560c5d68e756-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ebb6e26-92d4-4623-971c-560c5d68e756" (UID: "8ebb6e26-92d4-4623-971c-560c5d68e756"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:24:24.396036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.395991    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ebb6e26-92d4-4623-971c-560c5d68e756-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:24.963236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.963205    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"
Jul 18 01:24:24.963445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.963239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g" event={"ID":"8ebb6e26-92d4-4623-971c-560c5d68e756","Type":"ContainerDied","Data":"bcdc7b1f47c0ce09cac1ef5ad381d5d75bbd5dbeba4684b3f93bd126ada86dbd"}
Jul 18 01:24:24.963445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.963325    2567 scope.go:117] "RemoveContainer" containerID="3f62e0a8511f31283902b0e4145e10bd7e525feead705109501a42622f6b1c5d"
Jul 18 01:24:24.965850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.965825    2567 generic.go:356] "Generic (PLEG): container finished" podID="64980ad9-d542-47fd-ab0a-019291805fc3" containerID="98537baa07f7f67fc8aaa384c83b55ca353bea467e708e7c4d8e60b745808eb4" exitCode=6
Jul 18 01:24:24.965985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.965865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" event={"ID":"64980ad9-d542-47fd-ab0a-019291805fc3","Type":"ContainerDied","Data":"98537baa07f7f67fc8aaa384c83b55ca353bea467e708e7c4d8e60b745808eb4"}
Jul 18 01:24:24.966190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.966173    2567 scope.go:117] "RemoveContainer" containerID="98537baa07f7f67fc8aaa384c83b55ca353bea467e708e7c4d8e60b745808eb4"
Jul 18 01:24:24.966449 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:24.966427    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w_odh-ai-gateway-infra(64980ad9-d542-47fd-ab0a-019291805fc3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" podUID="64980ad9-d542-47fd-ab0a-019291805fc3"
Jul 18 01:24:24.972317 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.972303    2567 scope.go:117] "RemoveContainer" containerID="c864a1c29e5a056058b488225fadf1d5edbd45b15222a3ce9a2d55d230029134"
Jul 18 01:24:24.978613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.978589    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"]
Jul 18 01:24:24.980446 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:24.980420    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-6h76g"]
Jul 18 01:24:25.439757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:25.439727    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:25.540079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:25.540043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:25.913393 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:25.913368    2567 scope.go:117] "RemoveContainer" containerID="649bffc9695f936b652c1939bd75b46f5224f83e90e24f2331bcc5dc20a08233"
Jul 18 01:24:26.001598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.001558    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"]
Jul 18 01:24:26.139897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.139873    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:26.311259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.311229    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64980ad9-d542-47fd-ab0a-019291805fc3-maas-api-service-ca\") pod \"64980ad9-d542-47fd-ab0a-019291805fc3\" (UID: \"64980ad9-d542-47fd-ab0a-019291805fc3\") "
Jul 18 01:24:26.311545 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.311523    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64980ad9-d542-47fd-ab0a-019291805fc3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64980ad9-d542-47fd-ab0a-019291805fc3" (UID: "64980ad9-d542-47fd-ab0a-019291805fc3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:24:26.412767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.412723    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64980ad9-d542-47fd-ab0a-019291805fc3-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:26.919732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.919648    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" path="/var/lib/kubelet/pods/8ebb6e26-92d4-4623-971c-560c5d68e756/volumes"
Jul 18 01:24:26.980907 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.980874    2567 generic.go:356] "Generic (PLEG): container finished" podID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerID="49419434b1fa858cbf27ecb4711df77528feafd0a240051a44bf2ec82386ce00" exitCode=6
Jul 18 01:24:26.981097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.980954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" event={"ID":"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e","Type":"ContainerDied","Data":"49419434b1fa858cbf27ecb4711df77528feafd0a240051a44bf2ec82386ce00"}
Jul 18 01:24:26.981097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.981021    2567 scope.go:117] "RemoveContainer" containerID="649bffc9695f936b652c1939bd75b46f5224f83e90e24f2331bcc5dc20a08233"
Jul 18 01:24:26.981400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.981382    2567 scope.go:117] "RemoveContainer" containerID="49419434b1fa858cbf27ecb4711df77528feafd0a240051a44bf2ec82386ce00"
Jul 18 01:24:26.981654 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:26.981633    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq_odh-ai-gateway-infra(12cc9d2a-e2ba-4abf-a5f7-e73300645d6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e"
Jul 18 01:24:26.982399 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.982376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w" event={"ID":"64980ad9-d542-47fd-ab0a-019291805fc3","Type":"ContainerDied","Data":"4eefb652ee64e5b107c9e8bc61200ce66486608bb613a8007bc6e149cd394a3c"}
Jul 18 01:24:26.982482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.982385    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"
Jul 18 01:24:26.992551 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:26.992532    2567 scope.go:117] "RemoveContainer" containerID="98537baa07f7f67fc8aaa384c83b55ca353bea467e708e7c4d8e60b745808eb4"
Jul 18 01:24:27.007066 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:27.007040    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"]
Jul 18 01:24:27.012645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:27.012617    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-s6r2w"]
Jul 18 01:24:28.005501 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.005466    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"]
Jul 18 01:24:28.140099 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.140075    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:28.230662 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.230624    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e-maas-api-service-ca\") pod \"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e\" (UID: \"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e\") "
Jul 18 01:24:28.231040 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.231016    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" (UID: "12cc9d2a-e2ba-4abf-a5f7-e73300645d6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:24:28.331371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.331334    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:24:28.925437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.925403    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" path="/var/lib/kubelet/pods/64980ad9-d542-47fd-ab0a-019291805fc3/volumes"
Jul 18 01:24:28.997486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.997445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq" event={"ID":"12cc9d2a-e2ba-4abf-a5f7-e73300645d6e","Type":"ContainerDied","Data":"7c6403d9a8a7d4ba407955c9b1c60abdbd2a2c0f23ac4c10f6903ab96a92220a"}
Jul 18 01:24:28.997698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.997504    2567 scope.go:117] "RemoveContainer" containerID="49419434b1fa858cbf27ecb4711df77528feafd0a240051a44bf2ec82386ce00"
Jul 18 01:24:28.997698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:28.997463    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"
Jul 18 01:24:29.012936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:29.012900    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"]
Jul 18 01:24:29.017905 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:29.017881    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-5v6kq"]
Jul 18 01:24:30.919615 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:30.919580    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" path="/var/lib/kubelet/pods/12cc9d2a-e2ba-4abf-a5f7-e73300645d6e/volumes"
Jul 18 01:24:32.097801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.097764    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:32.415417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415337    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"]
Jul 18 01:24:32.415713 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415692    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415713 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415707    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415713 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415716    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:32.415713 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415721    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415729    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415735    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415745    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415750    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415760    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415766    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415774    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415781    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415789    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415794    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415801    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415806    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415812    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415817    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415824    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415828    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415886    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415892    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415899    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415903    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415909    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415917    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415923    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:32.415920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415928    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.416668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415934    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.416668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415940    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy"
Jul 18 01:24:32.416668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.415946    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:32.420452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.420435    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.423574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.423555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f92ba3bc-openshift-default-dockercfg-rj8mv\""
Jul 18 01:24:32.437463 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.437442    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"]
Jul 18 01:24:32.440422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.440398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:32.570847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.570813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-socket\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.570847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.570849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-credential-socket\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.570879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-envoy\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.570906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-certs\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.570930    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc9b192-e303-4938-935a-97229dede9bf-istio-podinfo\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.570952    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-data\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.571035    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6748\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-kube-api-access-v6748\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.571087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-istio-token\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.571145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.571106    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9b192-e303-4938-935a-97229dede9bf-istiod-ca-cert\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672062    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-istio-token\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9b192-e303-4938-935a-97229dede9bf-istiod-ca-cert\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-socket\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-credential-socket\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-envoy\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-certs\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc9b192-e303-4938-935a-97229dede9bf-istio-podinfo\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672486    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-socket\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-data\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6748\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-kube-api-access-v6748\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-credential-socket\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672704    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-certs\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.672891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.672771    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-data\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.673149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.673092    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9b192-e303-4938-935a-97229dede9bf-istiod-ca-cert\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.674861 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.674840    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-envoy\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.675202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.675180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc9b192-e303-4938-935a-97229dede9bf-istio-podinfo\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.685472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.685444    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-istio-token\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.690392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.690359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6748\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-kube-api-access-v6748\") pod \"e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") " pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.731115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.731077    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:32.884675 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.884644    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"]
Jul 18 01:24:32.887203 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:24:32.887168    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc9b192_e303_4938_935a_97229dede9bf.slice/crio-7c79bd4178892c765b5383cc1d49c46041a498b17d8c7af55023b362ae7dbbe0 WatchSource:0}: Error finding container 7c79bd4178892c765b5383cc1d49c46041a498b17d8c7af55023b362ae7dbbe0: Status 404 returned error can't find the container with id 7c79bd4178892c765b5383cc1d49c46041a498b17d8c7af55023b362ae7dbbe0
Jul 18 01:24:32.889259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.889227    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:24:32.889339 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.889311    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:24:32.889381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.889343    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:24:32.912488 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:32.912462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:33.030309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:33.024549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" event={"ID":"5cc9b192-e303-4938-935a-97229dede9bf","Type":"ContainerStarted","Data":"4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"}
Jul 18 01:24:33.030309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:33.024662    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" event={"ID":"5cc9b192-e303-4938-935a-97229dede9bf","Type":"ContainerStarted","Data":"7c79bd4178892c765b5383cc1d49c46041a498b17d8c7af55023b362ae7dbbe0"}
Jul 18 01:24:33.050116 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:33.050058    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" podStartSLOduration=1.050042198 podStartE2EDuration="1.050042198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:24:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:24:33.047705302 +0000 UTC m=+2652.575210263" watchObservedRunningTime="2026-07-18 01:24:33.050042198 +0000 UTC m=+2652.577547157"
Jul 18 01:24:33.731227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:33.731171    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:33.736316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:33.736286    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:34.030090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:34.029994    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:34.031065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:34.031039    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:34.944365 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:34.944333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:35.040420 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:35.040391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:35.140890 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:35.140856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:35.481565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:35.481529    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:35.591743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:35.591708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:35.699329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:35.699301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:37.540925 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:37.540890    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:24:39.397740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:39.397707    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:24:46.104421 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:46.104389    2567 generic.go:356] "Generic (PLEG): container finished" podID="5cc9b192-e303-4938-935a-97229dede9bf" containerID="4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13" exitCode=0
Jul 18 01:24:46.104963 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:46.104466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" event={"ID":"5cc9b192-e303-4938-935a-97229dede9bf","Type":"ContainerDied","Data":"4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"}
Jul 18 01:24:46.105041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:46.104964    2567 scope.go:117] "RemoveContainer" containerID="4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"
Jul 18 01:24:46.106090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:46.106058    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:24:46.106202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:46.106129    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:24:46.106202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:46.106164    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:24:47.112929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:47.112893    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" event={"ID":"5cc9b192-e303-4938-935a-97229dede9bf","Type":"ContainerStarted","Data":"1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969"}
Jul 18 01:24:47.731502 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:47.731468    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:47.732479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:47.732457    2567 patch_prober.go:28] interesting pod/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Jul 18 01:24:47.732545 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:47.732518    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Jul 18 01:24:48.732211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.732173    2567 patch_prober.go:28] interesting pod/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Jul 18 01:24:48.732651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.732246    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Jul 18 01:24:48.947445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947412    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"]
Jul 18 01:24:48.947780 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947767    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:48.947836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947781    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:48.947836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947792    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy"
Jul 18 01:24:48.947836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947798    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d630c29b-06df-4cff-8b2e-3844c4181f0c" containerName="istio-proxy"
Jul 18 01:24:48.947836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947811    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:48.947836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947817    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="64980ad9-d542-47fd-ab0a-019291805fc3" containerName="revoke-keys"
Jul 18 01:24:48.947836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947834    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:48.948050 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947839    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:48.948050 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947901    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="12cc9d2a-e2ba-4abf-a5f7-e73300645d6e" containerName="revoke-keys"
Jul 18 01:24:48.948050 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947909    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c874c0d8-ddc7-41b7-b4e6-60dbd497ce2c" containerName="revoke-keys"
Jul 18 01:24:48.948050 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.947918    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ebb6e26-92d4-4623-971c-560c5d68e756" containerName="revoke-keys"
Jul 18 01:24:48.951686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.951653    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:24:48.954613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.954578    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:24:48.954781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.954757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:24:48.958951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:48.958927    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"]
Jul 18 01:24:49.031111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.031016    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:49.031373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.031341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6df46aeb-bc04-47e5-bb95-72d78dd4e049-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-cml62\" (UID: \"6df46aeb-bc04-47e5-bb95-72d78dd4e049\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:24:49.132588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.132557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6df46aeb-bc04-47e5-bb95-72d78dd4e049-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-cml62\" (UID: \"6df46aeb-bc04-47e5-bb95-72d78dd4e049\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:24:49.133166 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.133147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6df46aeb-bc04-47e5-bb95-72d78dd4e049-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-cml62\" (UID: \"6df46aeb-bc04-47e5-bb95-72d78dd4e049\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:24:49.270569 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.270522    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:24:49.430317 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.430295    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"]
Jul 18 01:24:49.433039 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:24:49.433016    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df46aeb_bc04_47e5_bb95_72d78dd4e049.slice/crio-871bdfd75254860c03693a6451c6587e5fadd3adcd4799437178c6b2c3b8b931 WatchSource:0}: Error finding container 871bdfd75254860c03693a6451c6587e5fadd3adcd4799437178c6b2c3b8b931: Status 404 returned error can't find the container with id 871bdfd75254860c03693a6451c6587e5fadd3adcd4799437178c6b2c3b8b931
Jul 18 01:24:49.731781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.731742    2567 patch_prober.go:28] interesting pod/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Jul 18 01:24:49.731960 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:49.731798    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Jul 18 01:24:50.130839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:50.130802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" event={"ID":"6df46aeb-bc04-47e5-bb95-72d78dd4e049","Type":"ContainerStarted","Data":"e35d1c7f719c3b83939c92afc0469fec81b61853d37ac2f26a9783d7229c9949"}
Jul 18 01:24:50.131260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:50.130862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" event={"ID":"6df46aeb-bc04-47e5-bb95-72d78dd4e049","Type":"ContainerStarted","Data":"871bdfd75254860c03693a6451c6587e5fadd3adcd4799437178c6b2c3b8b931"}
Jul 18 01:24:50.144925 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:50.144875    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" podStartSLOduration=2.144859506 podStartE2EDuration="2.144859506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:24:50.144188423 +0000 UTC m=+2669.671693384" watchObservedRunningTime="2026-07-18 01:24:50.144859506 +0000 UTC m=+2669.672364466"
Jul 18 01:24:50.731945 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:50.731854    2567 patch_prober.go:28] interesting pod/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Jul 18 01:24:50.731945 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:50.731906    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Jul 18 01:24:51.137985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:51.137941    2567 generic.go:356] "Generic (PLEG): container finished" podID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerID="e35d1c7f719c3b83939c92afc0469fec81b61853d37ac2f26a9783d7229c9949" exitCode=6
Jul 18 01:24:51.138475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:51.138003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" event={"ID":"6df46aeb-bc04-47e5-bb95-72d78dd4e049","Type":"ContainerDied","Data":"e35d1c7f719c3b83939c92afc0469fec81b61853d37ac2f26a9783d7229c9949"}
Jul 18 01:24:51.138475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:51.138250    2567 scope.go:117] "RemoveContainer" containerID="e35d1c7f719c3b83939c92afc0469fec81b61853d37ac2f26a9783d7229c9949"
Jul 18 01:24:51.735717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:51.735636    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:51.736675 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:51.736655    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:24:52.144387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:52.144348    2567 generic.go:356] "Generic (PLEG): container finished" podID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerID="e3a3b81fe7b465dd41288ecda946fc05abf015198b9baf892ee5309d7114e0fa" exitCode=6
Jul 18 01:24:52.144847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:52.144423    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" event={"ID":"6df46aeb-bc04-47e5-bb95-72d78dd4e049","Type":"ContainerDied","Data":"e3a3b81fe7b465dd41288ecda946fc05abf015198b9baf892ee5309d7114e0fa"}
Jul 18 01:24:52.144847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:52.144479    2567 scope.go:117] "RemoveContainer" containerID="e35d1c7f719c3b83939c92afc0469fec81b61853d37ac2f26a9783d7229c9949"
Jul 18 01:24:52.144847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:52.144654    2567 scope.go:117] "RemoveContainer" containerID="e3a3b81fe7b465dd41288ecda946fc05abf015198b9baf892ee5309d7114e0fa"
Jul 18 01:24:52.144986 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:52.144887    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-cml62_odh-ai-gateway-infra(6df46aeb-bc04-47e5-bb95-72d78dd4e049)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049"
Jul 18 01:24:53.150860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:53.150836    2567 scope.go:117] "RemoveContainer" containerID="e3a3b81fe7b465dd41288ecda946fc05abf015198b9baf892ee5309d7114e0fa"
Jul 18 01:24:53.151285 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:24:53.151058    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-cml62_odh-ai-gateway-infra(6df46aeb-bc04-47e5-bb95-72d78dd4e049)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049"
Jul 18 01:24:59.444930 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:24:59.444893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:25:01.088683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.088643    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"]
Jul 18 01:25:01.092015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.091998    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:01.098919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.098884    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"]
Jul 18 01:25:01.249935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.249896    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ab5f72-800a-401c-a73a-9f9a1899e412-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx\" (UID: \"34ab5f72-800a-401c-a73a-9f9a1899e412\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:01.350862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.350766    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ab5f72-800a-401c-a73a-9f9a1899e412-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx\" (UID: \"34ab5f72-800a-401c-a73a-9f9a1899e412\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:01.351572 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.351548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ab5f72-800a-401c-a73a-9f9a1899e412-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx\" (UID: \"34ab5f72-800a-401c-a73a-9f9a1899e412\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:01.404752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.404700    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:01.569906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:01.569755    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"]
Jul 18 01:25:01.572805 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:25:01.572776    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ab5f72_800a_401c_a73a_9f9a1899e412.slice/crio-6d9b1f762350f0cfd84b5f66efea1eb54f83c82433d60936de29b31c2b34f9ad WatchSource:0}: Error finding container 6d9b1f762350f0cfd84b5f66efea1eb54f83c82433d60936de29b31c2b34f9ad: Status 404 returned error can't find the container with id 6d9b1f762350f0cfd84b5f66efea1eb54f83c82433d60936de29b31c2b34f9ad
Jul 18 01:25:02.202007 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:02.201917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerStarted","Data":"d193b72d045afd5c2ef9cca66722d3f2b22f90a78d5a103f72742a4dabc8b3cd"}
Jul 18 01:25:02.202007 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:02.201975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerStarted","Data":"6d9b1f762350f0cfd84b5f66efea1eb54f83c82433d60936de29b31c2b34f9ad"}
Jul 18 01:25:02.217188 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:02.217139    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" podStartSLOduration=1.21712254 podStartE2EDuration="1.21712254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:25:02.216107124 +0000 UTC m=+2681.743612084" watchObservedRunningTime="2026-07-18 01:25:02.21712254 +0000 UTC m=+2681.744627499"
Jul 18 01:25:03.111836 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.111805    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:25:03.208559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.208520    2567 generic.go:356] "Generic (PLEG): container finished" podID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerID="d193b72d045afd5c2ef9cca66722d3f2b22f90a78d5a103f72742a4dabc8b3cd" exitCode=6
Jul 18 01:25:03.208953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.208563    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerDied","Data":"d193b72d045afd5c2ef9cca66722d3f2b22f90a78d5a103f72742a4dabc8b3cd"}
Jul 18 01:25:03.208953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.208798    2567 scope.go:117] "RemoveContainer" containerID="d193b72d045afd5c2ef9cca66722d3f2b22f90a78d5a103f72742a4dabc8b3cd"
Jul 18 01:25:03.555382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.555337    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"]
Jul 18 01:25:03.559006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.558982    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:03.565758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.565726    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"]
Jul 18 01:25:03.671780 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.671680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e30e98ff-0421-475b-9cbb-4b6823ac8fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj\" (UID: \"e30e98ff-0421-475b-9cbb-4b6823ac8fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:03.772653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.772609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e30e98ff-0421-475b-9cbb-4b6823ac8fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj\" (UID: \"e30e98ff-0421-475b-9cbb-4b6823ac8fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:03.773253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.773233    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e30e98ff-0421-475b-9cbb-4b6823ac8fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj\" (UID: \"e30e98ff-0421-475b-9cbb-4b6823ac8fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:03.870519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:03.870489    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:04.013529 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:04.013502    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"]
Jul 18 01:25:04.016107 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:25:04.016080    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30e98ff_0421_475b_9cbb_4b6823ac8fbd.slice/crio-2c6fc682ac6997bd161017cfff2c6cf213b9133935070e30de6dcb3d009bd53f WatchSource:0}: Error finding container 2c6fc682ac6997bd161017cfff2c6cf213b9133935070e30de6dcb3d009bd53f: Status 404 returned error can't find the container with id 2c6fc682ac6997bd161017cfff2c6cf213b9133935070e30de6dcb3d009bd53f
Jul 18 01:25:04.215992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:04.215908    2567 generic.go:356] "Generic (PLEG): container finished" podID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerID="3186b554f1c08c811e00e196411200077dd6581c1cd7eeac7204022b5d0be3ae" exitCode=6
Jul 18 01:25:04.216496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:04.215988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerDied","Data":"3186b554f1c08c811e00e196411200077dd6581c1cd7eeac7204022b5d0be3ae"}
Jul 18 01:25:04.216496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:04.216062    2567 scope.go:117] "RemoveContainer" containerID="d193b72d045afd5c2ef9cca66722d3f2b22f90a78d5a103f72742a4dabc8b3cd"
Jul 18 01:25:04.216496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:04.216338    2567 scope.go:117] "RemoveContainer" containerID="3186b554f1c08c811e00e196411200077dd6581c1cd7eeac7204022b5d0be3ae"
Jul 18 01:25:04.216677 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:04.216572    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx_odh-ai-gateway-infra(34ab5f72-800a-401c-a73a-9f9a1899e412)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412"
Jul 18 01:25:04.217484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:04.217461    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" event={"ID":"e30e98ff-0421-475b-9cbb-4b6823ac8fbd","Type":"ContainerStarted","Data":"2c6fc682ac6997bd161017cfff2c6cf213b9133935070e30de6dcb3d009bd53f"}
Jul 18 01:25:05.225044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:05.225011    2567 scope.go:117] "RemoveContainer" containerID="3186b554f1c08c811e00e196411200077dd6581c1cd7eeac7204022b5d0be3ae"
Jul 18 01:25:05.225534 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:05.225308    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx_odh-ai-gateway-infra(34ab5f72-800a-401c-a73a-9f9a1899e412)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412"
Jul 18 01:25:05.226410 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:05.226387    2567 generic.go:356] "Generic (PLEG): container finished" podID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerID="5ad4d0bb47678e061616dd5c983fbf331a0bbc311bce29c7b3dcbb00205751ab" exitCode=6
Jul 18 01:25:05.226504 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:05.226463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" event={"ID":"e30e98ff-0421-475b-9cbb-4b6823ac8fbd","Type":"ContainerDied","Data":"5ad4d0bb47678e061616dd5c983fbf331a0bbc311bce29c7b3dcbb00205751ab"}
Jul 18 01:25:05.226603 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:05.226588    2567 scope.go:117] "RemoveContainer" containerID="5ad4d0bb47678e061616dd5c983fbf331a0bbc311bce29c7b3dcbb00205751ab"
Jul 18 01:25:06.155465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.155367    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"]
Jul 18 01:25:06.158874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.158847    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:06.165294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.165251    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"]
Jul 18 01:25:06.233801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.233755    2567 generic.go:356] "Generic (PLEG): container finished" podID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerID="1db31a460c4151544412dd97191909f613b2166eda585795358aba2031979aab" exitCode=6
Jul 18 01:25:06.234218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.233837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" event={"ID":"e30e98ff-0421-475b-9cbb-4b6823ac8fbd","Type":"ContainerDied","Data":"1db31a460c4151544412dd97191909f613b2166eda585795358aba2031979aab"}
Jul 18 01:25:06.234218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.233901    2567 scope.go:117] "RemoveContainer" containerID="5ad4d0bb47678e061616dd5c983fbf331a0bbc311bce29c7b3dcbb00205751ab"
Jul 18 01:25:06.234218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.234073    2567 scope.go:117] "RemoveContainer" containerID="1db31a460c4151544412dd97191909f613b2166eda585795358aba2031979aab"
Jul 18 01:25:06.234418 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:06.234357    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj_odh-ai-gateway-infra(e30e98ff-0421-475b-9cbb-4b6823ac8fbd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd"
Jul 18 01:25:06.294577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.294535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa65010-cbf9-498e-a4d3-a1882cbfc127-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq\" (UID: \"baa65010-cbf9-498e-a4d3-a1882cbfc127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:06.395837 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.395792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa65010-cbf9-498e-a4d3-a1882cbfc127-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq\" (UID: \"baa65010-cbf9-498e-a4d3-a1882cbfc127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:06.396435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.396414    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa65010-cbf9-498e-a4d3-a1882cbfc127-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq\" (UID: \"baa65010-cbf9-498e-a4d3-a1882cbfc127\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:06.471800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.471705    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:06.614048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:06.614024    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"]
Jul 18 01:25:06.615625 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:25:06.615598    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa65010_cbf9_498e_a4d3_a1882cbfc127.slice/crio-65a356e2ca8ec0ccbcb9b5cfa7359c2ec33c436aa3492b0137e61c5d35a48d8b WatchSource:0}: Error finding container 65a356e2ca8ec0ccbcb9b5cfa7359c2ec33c436aa3492b0137e61c5d35a48d8b: Status 404 returned error can't find the container with id 65a356e2ca8ec0ccbcb9b5cfa7359c2ec33c436aa3492b0137e61c5d35a48d8b
Jul 18 01:25:07.240631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:07.240526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerStarted","Data":"1b40754b2b0add026d780f93a52cf29472a9d29c3380c801db4ef3f58095402b"}
Jul 18 01:25:07.240631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:07.240588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerStarted","Data":"65a356e2ca8ec0ccbcb9b5cfa7359c2ec33c436aa3492b0137e61c5d35a48d8b"}
Jul 18 01:25:07.242346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:07.242329    2567 scope.go:117] "RemoveContainer" containerID="1db31a460c4151544412dd97191909f613b2166eda585795358aba2031979aab"
Jul 18 01:25:07.242506 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:07.242486    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj_odh-ai-gateway-infra(e30e98ff-0421-475b-9cbb-4b6823ac8fbd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd"
Jul 18 01:25:07.255321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:07.255252    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" podStartSLOduration=1.255237775 podStartE2EDuration="1.255237775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:25:07.254400704 +0000 UTC m=+2686.781905666" watchObservedRunningTime="2026-07-18 01:25:07.255237775 +0000 UTC m=+2686.782742734"
Jul 18 01:25:07.604532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:07.604482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:25:07.705909 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:07.705877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:25:08.253189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:08.253137    2567 generic.go:356] "Generic (PLEG): container finished" podID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerID="1b40754b2b0add026d780f93a52cf29472a9d29c3380c801db4ef3f58095402b" exitCode=6
Jul 18 01:25:08.253592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:08.253211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerDied","Data":"1b40754b2b0add026d780f93a52cf29472a9d29c3380c801db4ef3f58095402b"}
Jul 18 01:25:08.253592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:08.253462    2567 scope.go:117] "RemoveContainer" containerID="1b40754b2b0add026d780f93a52cf29472a9d29c3380c801db4ef3f58095402b"
Jul 18 01:25:08.913976 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:08.913946    2567 scope.go:117] "RemoveContainer" containerID="e3a3b81fe7b465dd41288ecda946fc05abf015198b9baf892ee5309d7114e0fa"
Jul 18 01:25:09.261426 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:09.261398    2567 generic.go:356] "Generic (PLEG): container finished" podID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerID="69fdd53bd54b924601fe6708f6f3f0a9e677ee95eb4be71115920e4fd50aedf7" exitCode=6
Jul 18 01:25:09.261851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:09.261460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerDied","Data":"69fdd53bd54b924601fe6708f6f3f0a9e677ee95eb4be71115920e4fd50aedf7"}
Jul 18 01:25:09.261851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:09.261501    2567 scope.go:117] "RemoveContainer" containerID="1b40754b2b0add026d780f93a52cf29472a9d29c3380c801db4ef3f58095402b"
Jul 18 01:25:09.261851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:09.261732    2567 scope.go:117] "RemoveContainer" containerID="69fdd53bd54b924601fe6708f6f3f0a9e677ee95eb4be71115920e4fd50aedf7"
Jul 18 01:25:09.262053 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:09.262018    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq_odh-ai-gateway-infra(baa65010-cbf9-498e-a4d3-a1882cbfc127)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127"
Jul 18 01:25:10.269317 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:10.269255    2567 generic.go:356] "Generic (PLEG): container finished" podID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerID="607310c6a2aaa4401260da3bb729b8fb3838a129be685a0979385ce44f68bb85" exitCode=6
Jul 18 01:25:10.269764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:10.269331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" event={"ID":"6df46aeb-bc04-47e5-bb95-72d78dd4e049","Type":"ContainerDied","Data":"607310c6a2aaa4401260da3bb729b8fb3838a129be685a0979385ce44f68bb85"}
Jul 18 01:25:10.269764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:10.269391    2567 scope.go:117] "RemoveContainer" containerID="e3a3b81fe7b465dd41288ecda946fc05abf015198b9baf892ee5309d7114e0fa"
Jul 18 01:25:10.269764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:10.269685    2567 scope.go:117] "RemoveContainer" containerID="607310c6a2aaa4401260da3bb729b8fb3838a129be685a0979385ce44f68bb85"
Jul 18 01:25:10.269988 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:10.269924    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-cml62_odh-ai-gateway-infra(6df46aeb-bc04-47e5-bb95-72d78dd4e049)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049"
Jul 18 01:25:10.271301 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:10.271259    2567 scope.go:117] "RemoveContainer" containerID="69fdd53bd54b924601fe6708f6f3f0a9e677ee95eb4be71115920e4fd50aedf7"
Jul 18 01:25:10.271476 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:10.271460    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq_odh-ai-gateway-infra(baa65010-cbf9-498e-a4d3-a1882cbfc127)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127"
Jul 18 01:25:11.295995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:11.295902    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"]
Jul 18 01:25:11.441001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:11.440976    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:25:11.545030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:11.544996    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6df46aeb-bc04-47e5-bb95-72d78dd4e049-maas-api-service-ca\") pod \"6df46aeb-bc04-47e5-bb95-72d78dd4e049\" (UID: \"6df46aeb-bc04-47e5-bb95-72d78dd4e049\") "
Jul 18 01:25:11.545445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:11.545418    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6df46aeb-bc04-47e5-bb95-72d78dd4e049-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6df46aeb-bc04-47e5-bb95-72d78dd4e049" (UID: "6df46aeb-bc04-47e5-bb95-72d78dd4e049"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:25:11.645617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:11.645591    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6df46aeb-bc04-47e5-bb95-72d78dd4e049-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:25:12.284627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:12.284596    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"
Jul 18 01:25:12.284833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:12.284634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62" event={"ID":"6df46aeb-bc04-47e5-bb95-72d78dd4e049","Type":"ContainerDied","Data":"871bdfd75254860c03693a6451c6587e5fadd3adcd4799437178c6b2c3b8b931"}
Jul 18 01:25:12.284833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:12.284695    2567 scope.go:117] "RemoveContainer" containerID="607310c6a2aaa4401260da3bb729b8fb3838a129be685a0979385ce44f68bb85"
Jul 18 01:25:12.308989 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:12.308959    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"]
Jul 18 01:25:12.315326 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:12.315303    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-cml62"]
Jul 18 01:25:12.919351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:12.919318    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" path="/var/lib/kubelet/pods/6df46aeb-bc04-47e5-bb95-72d78dd4e049/volumes"
Jul 18 01:25:17.913476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:17.913448    2567 scope.go:117] "RemoveContainer" containerID="3186b554f1c08c811e00e196411200077dd6581c1cd7eeac7204022b5d0be3ae"
Jul 18 01:25:18.318375 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:18.318339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerStarted","Data":"8e67e0d650af1016c1b8a502367dcd0f234e3f55d1e46fc4f5efeccd1524e66c"}
Jul 18 01:25:18.913225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:18.913140    2567 scope.go:117] "RemoveContainer" containerID="1db31a460c4151544412dd97191909f613b2166eda585795358aba2031979aab"
Jul 18 01:25:19.326136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.326102    2567 generic.go:356] "Generic (PLEG): container finished" podID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerID="8e67e0d650af1016c1b8a502367dcd0f234e3f55d1e46fc4f5efeccd1524e66c" exitCode=6
Jul 18 01:25:19.326633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.326187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerDied","Data":"8e67e0d650af1016c1b8a502367dcd0f234e3f55d1e46fc4f5efeccd1524e66c"}
Jul 18 01:25:19.326633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.326318    2567 scope.go:117] "RemoveContainer" containerID="3186b554f1c08c811e00e196411200077dd6581c1cd7eeac7204022b5d0be3ae"
Jul 18 01:25:19.326633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.326611    2567 scope.go:117] "RemoveContainer" containerID="8e67e0d650af1016c1b8a502367dcd0f234e3f55d1e46fc4f5efeccd1524e66c"
Jul 18 01:25:19.326874 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:19.326855    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx_odh-ai-gateway-infra(34ab5f72-800a-401c-a73a-9f9a1899e412)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412"
Jul 18 01:25:19.328129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.328110    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" event={"ID":"e30e98ff-0421-475b-9cbb-4b6823ac8fbd","Type":"ContainerStarted","Data":"66b9fe590e9082969399cb6ae5f9efeaecdb1f153cfe7485121e987dce784daf"}
Jul 18 01:25:19.342893 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.342868    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"]
Jul 18 01:25:19.366794 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:19.366733    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" podStartSLOduration=16.366714354 podStartE2EDuration="16.366714354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:25:19.365388217 +0000 UTC m=+2698.892893178" watchObservedRunningTime="2026-07-18 01:25:19.366714354 +0000 UTC m=+2698.894219313"
Jul 18 01:25:20.336794 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.336760    2567 generic.go:356] "Generic (PLEG): container finished" podID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerID="66b9fe590e9082969399cb6ae5f9efeaecdb1f153cfe7485121e987dce784daf" exitCode=6
Jul 18 01:25:20.337153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.336843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" event={"ID":"e30e98ff-0421-475b-9cbb-4b6823ac8fbd","Type":"ContainerDied","Data":"66b9fe590e9082969399cb6ae5f9efeaecdb1f153cfe7485121e987dce784daf"}
Jul 18 01:25:20.337153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.336922    2567 scope.go:117] "RemoveContainer" containerID="1db31a460c4151544412dd97191909f613b2166eda585795358aba2031979aab"
Jul 18 01:25:20.337289 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.337246    2567 scope.go:117] "RemoveContainer" containerID="66b9fe590e9082969399cb6ae5f9efeaecdb1f153cfe7485121e987dce784daf"
Jul 18 01:25:20.337489 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:20.337468    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj_odh-ai-gateway-infra(e30e98ff-0421-475b-9cbb-4b6823ac8fbd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd"
Jul 18 01:25:20.374430 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.374351    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"]
Jul 18 01:25:20.483572 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.483544    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:20.525692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.525656    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ab5f72-800a-401c-a73a-9f9a1899e412-maas-api-service-ca\") pod \"34ab5f72-800a-401c-a73a-9f9a1899e412\" (UID: \"34ab5f72-800a-401c-a73a-9f9a1899e412\") "
Jul 18 01:25:20.526019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.525995    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34ab5f72-800a-401c-a73a-9f9a1899e412-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "34ab5f72-800a-401c-a73a-9f9a1899e412" (UID: "34ab5f72-800a-401c-a73a-9f9a1899e412"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:25:20.626337 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:20.626242    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34ab5f72-800a-401c-a73a-9f9a1899e412-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:25:21.070550 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.070515    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:25:21.079034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.079005    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:25:21.344792 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.344760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx" event={"ID":"34ab5f72-800a-401c-a73a-9f9a1899e412","Type":"ContainerDied","Data":"6d9b1f762350f0cfd84b5f66efea1eb54f83c82433d60936de29b31c2b34f9ad"}
Jul 18 01:25:21.345164 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.344821    2567 scope.go:117] "RemoveContainer" containerID="8e67e0d650af1016c1b8a502367dcd0f234e3f55d1e46fc4f5efeccd1524e66c"
Jul 18 01:25:21.345164 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.344782    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"
Jul 18 01:25:21.376455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.376421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"]
Jul 18 01:25:21.380145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.380114    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-wbspx"]
Jul 18 01:25:21.489482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.489456    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:21.533498 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.533467    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e30e98ff-0421-475b-9cbb-4b6823ac8fbd-maas-api-service-ca\") pod \"e30e98ff-0421-475b-9cbb-4b6823ac8fbd\" (UID: \"e30e98ff-0421-475b-9cbb-4b6823ac8fbd\") "
Jul 18 01:25:21.533943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.533913    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e30e98ff-0421-475b-9cbb-4b6823ac8fbd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e30e98ff-0421-475b-9cbb-4b6823ac8fbd" (UID: "e30e98ff-0421-475b-9cbb-4b6823ac8fbd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:25:21.612159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.612080    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:25:21.634747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:21.634710    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e30e98ff-0421-475b-9cbb-4b6823ac8fbd-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:25:22.351448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.351411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj" event={"ID":"e30e98ff-0421-475b-9cbb-4b6823ac8fbd","Type":"ContainerDied","Data":"2c6fc682ac6997bd161017cfff2c6cf213b9133935070e30de6dcb3d009bd53f"}
Jul 18 01:25:22.351906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.351426    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"
Jul 18 01:25:22.351906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.351523    2567 scope.go:117] "RemoveContainer" containerID="66b9fe590e9082969399cb6ae5f9efeaecdb1f153cfe7485121e987dce784daf"
Jul 18 01:25:22.375194 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.375156    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"]
Jul 18 01:25:22.379764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.379730    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-q5zvj"]
Jul 18 01:25:22.920435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.920403    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" path="/var/lib/kubelet/pods/34ab5f72-800a-401c-a73a-9f9a1899e412/volumes"
Jul 18 01:25:22.920776 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:22.920763    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" path="/var/lib/kubelet/pods/e30e98ff-0421-475b-9cbb-4b6823ac8fbd/volumes"
Jul 18 01:25:23.913503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:23.913477    2567 scope.go:117] "RemoveContainer" containerID="69fdd53bd54b924601fe6708f6f3f0a9e677ee95eb4be71115920e4fd50aedf7"
Jul 18 01:25:24.367757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:24.367720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerStarted","Data":"b6626867948dec60fcaeeebef3ba44ec2694f6b4d2c55f9e37471a17334f3ec9"}
Jul 18 01:25:25.374534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:25.374497    2567 generic.go:356] "Generic (PLEG): container finished" podID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerID="b6626867948dec60fcaeeebef3ba44ec2694f6b4d2c55f9e37471a17334f3ec9" exitCode=6
Jul 18 01:25:25.374988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:25.374571    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerDied","Data":"b6626867948dec60fcaeeebef3ba44ec2694f6b4d2c55f9e37471a17334f3ec9"}
Jul 18 01:25:25.374988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:25.374633    2567 scope.go:117] "RemoveContainer" containerID="69fdd53bd54b924601fe6708f6f3f0a9e677ee95eb4be71115920e4fd50aedf7"
Jul 18 01:25:25.374988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:25.374937    2567 scope.go:117] "RemoveContainer" containerID="b6626867948dec60fcaeeebef3ba44ec2694f6b4d2c55f9e37471a17334f3ec9"
Jul 18 01:25:25.375150 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:25.375130    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq_odh-ai-gateway-infra(baa65010-cbf9-498e-a4d3-a1882cbfc127)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127"
Jul 18 01:25:25.398903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:25.398862    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"]
Jul 18 01:25:26.524985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:26.524960    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:26.582802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:26.582767    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa65010-cbf9-498e-a4d3-a1882cbfc127-maas-api-service-ca\") pod \"baa65010-cbf9-498e-a4d3-a1882cbfc127\" (UID: \"baa65010-cbf9-498e-a4d3-a1882cbfc127\") "
Jul 18 01:25:26.583167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:26.583135    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baa65010-cbf9-498e-a4d3-a1882cbfc127-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "baa65010-cbf9-498e-a4d3-a1882cbfc127" (UID: "baa65010-cbf9-498e-a4d3-a1882cbfc127"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:25:26.684014 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:26.683900    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa65010-cbf9-498e-a4d3-a1882cbfc127-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:25:27.387262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.387235    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"
Jul 18 01:25:27.387486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.387297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq" event={"ID":"baa65010-cbf9-498e-a4d3-a1882cbfc127","Type":"ContainerDied","Data":"65a356e2ca8ec0ccbcb9b5cfa7359c2ec33c436aa3492b0137e61c5d35a48d8b"}
Jul 18 01:25:27.387486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.387361    2567 scope.go:117] "RemoveContainer" containerID="b6626867948dec60fcaeeebef3ba44ec2694f6b4d2c55f9e37471a17334f3ec9"
Jul 18 01:25:27.402765 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.402741    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"]
Jul 18 01:25:27.413468 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.413440    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-wz5bq"]
Jul 18 01:25:27.701751 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.701666    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"]
Jul 18 01:25:27.702227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702206    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702230    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702243    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702251    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702262    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702291    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702302    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702310    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702328    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.702346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702336    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702358    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702367    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702389    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702397    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702409    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702418    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702428    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702435    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702538    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702549    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702559    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702569    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702577    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702587    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702598    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702606    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702615    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:27.702773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.702625    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:27.705870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.705848    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:27.707988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.707968    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:25:27.708086 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.708007    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:25:27.712167 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.712144    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"]
Jul 18 01:25:27.794387 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.794351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d717c6f4-08b7-4fbf-a7a4-26672223aa2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9\" (UID: \"d717c6f4-08b7-4fbf-a7a4-26672223aa2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:27.895838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.895801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d717c6f4-08b7-4fbf-a7a4-26672223aa2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9\" (UID: \"d717c6f4-08b7-4fbf-a7a4-26672223aa2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:27.896596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:27.896572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d717c6f4-08b7-4fbf-a7a4-26672223aa2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9\" (UID: \"d717c6f4-08b7-4fbf-a7a4-26672223aa2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:28.018477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:28.018373    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:28.159901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:28.159876    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"]
Jul 18 01:25:28.162391 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:25:28.162356    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd717c6f4_08b7_4fbf_a7a4_26672223aa2a.slice/crio-e50721c481b202a17ac942085bf2734a26a767a9c72e6032432df09a55c774a2 WatchSource:0}: Error finding container e50721c481b202a17ac942085bf2734a26a767a9c72e6032432df09a55c774a2: Status 404 returned error can't find the container with id e50721c481b202a17ac942085bf2734a26a767a9c72e6032432df09a55c774a2
Jul 18 01:25:28.394830 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:28.394796    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9" event={"ID":"d717c6f4-08b7-4fbf-a7a4-26672223aa2a","Type":"ContainerStarted","Data":"e50721c481b202a17ac942085bf2734a26a767a9c72e6032432df09a55c774a2"}
Jul 18 01:25:28.921852 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:28.921823    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baa65010-cbf9-498e-a4d3-a1882cbfc127" path="/var/lib/kubelet/pods/baa65010-cbf9-498e-a4d3-a1882cbfc127/volumes"
Jul 18 01:25:29.401645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:29.401612    2567 generic.go:356] "Generic (PLEG): container finished" podID="d717c6f4-08b7-4fbf-a7a4-26672223aa2a" containerID="61e2a56483633ebec20b0712bf26397aede99aa12f2a25c90e0c63e40389ef8b" exitCode=0
Jul 18 01:25:29.401819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:29.401699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9" event={"ID":"d717c6f4-08b7-4fbf-a7a4-26672223aa2a","Type":"ContainerDied","Data":"61e2a56483633ebec20b0712bf26397aede99aa12f2a25c90e0c63e40389ef8b"}
Jul 18 01:25:30.544242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:30.544211    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:30.619389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:30.619355    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d717c6f4-08b7-4fbf-a7a4-26672223aa2a-maas-api-service-ca\") pod \"d717c6f4-08b7-4fbf-a7a4-26672223aa2a\" (UID: \"d717c6f4-08b7-4fbf-a7a4-26672223aa2a\") "
Jul 18 01:25:30.619733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:30.619708    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d717c6f4-08b7-4fbf-a7a4-26672223aa2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d717c6f4-08b7-4fbf-a7a4-26672223aa2a" (UID: "d717c6f4-08b7-4fbf-a7a4-26672223aa2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:25:30.720259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:30.720165    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d717c6f4-08b7-4fbf-a7a4-26672223aa2a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:25:31.413465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:31.413433    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9"
Jul 18 01:25:31.413663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:31.413475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9" event={"ID":"d717c6f4-08b7-4fbf-a7a4-26672223aa2a","Type":"ContainerDied","Data":"e50721c481b202a17ac942085bf2734a26a767a9c72e6032432df09a55c774a2"}
Jul 18 01:25:31.413663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:31.413530    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e50721c481b202a17ac942085bf2734a26a767a9c72e6032432df09a55c774a2"
Jul 18 01:25:38.228693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:38.228659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:25:42.107216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:42.107174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:25:42.798499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:42.798452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:25:47.809367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:47.809320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:25:48.986655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.986622    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"]
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.986989    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d717c6f4-08b7-4fbf-a7a4-26672223aa2a" containerName="revoke-keys"
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987000    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d717c6f4-08b7-4fbf-a7a4-26672223aa2a" containerName="revoke-keys"
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987009    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987015    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987023    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987028    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:48.987036 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987038    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:48.987256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987043    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e30e98ff-0421-475b-9cbb-4b6823ac8fbd" containerName="revoke-keys"
Jul 18 01:25:48.987256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987105    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df46aeb-bc04-47e5-bb95-72d78dd4e049" containerName="revoke-keys"
Jul 18 01:25:48.987256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987114    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d717c6f4-08b7-4fbf-a7a4-26672223aa2a" containerName="revoke-keys"
Jul 18 01:25:48.987256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.987122    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ab5f72-800a-401c-a73a-9f9a1899e412" containerName="revoke-keys"
Jul 18 01:25:48.990486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.990467    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:25:48.992902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.992881    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:25:48.993037 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.992903    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:25:48.997813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:48.997785    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"]
Jul 18 01:25:49.089021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:49.088972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f49001-9106-427e-a6c1-05fbcb58eb5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84\" (UID: \"90f49001-9106-427e-a6c1-05fbcb58eb5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:25:49.189693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:49.189652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f49001-9106-427e-a6c1-05fbcb58eb5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84\" (UID: \"90f49001-9106-427e-a6c1-05fbcb58eb5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:25:49.190327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:49.190306    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f49001-9106-427e-a6c1-05fbcb58eb5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84\" (UID: \"90f49001-9106-427e-a6c1-05fbcb58eb5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:25:49.303219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:49.303124    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:25:49.449310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:49.449261    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"]
Jul 18 01:25:49.451483 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:25:49.451455    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f49001_9106_427e_a6c1_05fbcb58eb5f.slice/crio-7527c4ae450dd60945d392a5b88aad326d01bef88b868f6708081c83ff22ec7e WatchSource:0}: Error finding container 7527c4ae450dd60945d392a5b88aad326d01bef88b868f6708081c83ff22ec7e: Status 404 returned error can't find the container with id 7527c4ae450dd60945d392a5b88aad326d01bef88b868f6708081c83ff22ec7e
Jul 18 01:25:49.509769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:49.509733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" event={"ID":"90f49001-9106-427e-a6c1-05fbcb58eb5f","Type":"ContainerStarted","Data":"7527c4ae450dd60945d392a5b88aad326d01bef88b868f6708081c83ff22ec7e"}
Jul 18 01:25:50.516647 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:50.516553    2567 generic.go:356] "Generic (PLEG): container finished" podID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerID="43edba35760df6a318f47b25d4ae71ff5358145b2ee6f003dc96fa276955cd44" exitCode=6
Jul 18 01:25:50.517022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:50.516637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" event={"ID":"90f49001-9106-427e-a6c1-05fbcb58eb5f","Type":"ContainerDied","Data":"43edba35760df6a318f47b25d4ae71ff5358145b2ee6f003dc96fa276955cd44"}
Jul 18 01:25:50.517022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:50.516775    2567 scope.go:117] "RemoveContainer" containerID="43edba35760df6a318f47b25d4ae71ff5358145b2ee6f003dc96fa276955cd44"
Jul 18 01:25:51.523385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:51.523297    2567 generic.go:356] "Generic (PLEG): container finished" podID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerID="e4bae6c08b3f6e29b7a41524dd52b02bb9042f5c1bde9af44226f0d5dd57dcf1" exitCode=6
Jul 18 01:25:51.523385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:51.523354    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" event={"ID":"90f49001-9106-427e-a6c1-05fbcb58eb5f","Type":"ContainerDied","Data":"e4bae6c08b3f6e29b7a41524dd52b02bb9042f5c1bde9af44226f0d5dd57dcf1"}
Jul 18 01:25:51.523829 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:51.523404    2567 scope.go:117] "RemoveContainer" containerID="43edba35760df6a318f47b25d4ae71ff5358145b2ee6f003dc96fa276955cd44"
Jul 18 01:25:51.523829 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:51.523551    2567 scope.go:117] "RemoveContainer" containerID="e4bae6c08b3f6e29b7a41524dd52b02bb9042f5c1bde9af44226f0d5dd57dcf1"
Jul 18 01:25:51.523829 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:51.523815    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84_odh-ai-gateway-infra(90f49001-9106-427e-a6c1-05fbcb58eb5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f"
Jul 18 01:25:52.529916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:52.529886    2567 scope.go:117] "RemoveContainer" containerID="e4bae6c08b3f6e29b7a41524dd52b02bb9042f5c1bde9af44226f0d5dd57dcf1"
Jul 18 01:25:52.530366 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:52.530090    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84_odh-ai-gateway-infra(90f49001-9106-427e-a6c1-05fbcb58eb5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f"
Jul 18 01:25:53.599378 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.599336    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"]
Jul 18 01:25:53.604150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.604130    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:25:53.609971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.609938    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"]
Jul 18 01:25:53.731826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.731783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/642992f9-8bcc-471f-bca6-d620a683d03b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-45wxj\" (UID: \"642992f9-8bcc-471f-bca6-d620a683d03b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:25:53.833041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.832996    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/642992f9-8bcc-471f-bca6-d620a683d03b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-45wxj\" (UID: \"642992f9-8bcc-471f-bca6-d620a683d03b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:25:53.833669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.833648    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/642992f9-8bcc-471f-bca6-d620a683d03b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-45wxj\" (UID: \"642992f9-8bcc-471f-bca6-d620a683d03b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:25:53.917400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:53.917287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:25:54.059698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:54.059674    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"]
Jul 18 01:25:54.061581 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:25:54.061554    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642992f9_8bcc_471f_bca6_d620a683d03b.slice/crio-352034cdefc94b93c514ade5c346058a1c28a4013cd9df7a8acb42813d6de920 WatchSource:0}: Error finding container 352034cdefc94b93c514ade5c346058a1c28a4013cd9df7a8acb42813d6de920: Status 404 returned error can't find the container with id 352034cdefc94b93c514ade5c346058a1c28a4013cd9df7a8acb42813d6de920
Jul 18 01:25:54.541543 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:54.541448    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" event={"ID":"642992f9-8bcc-471f-bca6-d620a683d03b","Type":"ContainerStarted","Data":"ba34079bc6b67c2dcba5bc0a990f9c8eb32730408874472189232d1b2bcdedc4"}
Jul 18 01:25:54.541543 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:54.541501    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" event={"ID":"642992f9-8bcc-471f-bca6-d620a683d03b","Type":"ContainerStarted","Data":"352034cdefc94b93c514ade5c346058a1c28a4013cd9df7a8acb42813d6de920"}
Jul 18 01:25:54.557081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:54.557018    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" podStartSLOduration=1.557001202 podStartE2EDuration="1.557001202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:25:54.5553211 +0000 UTC m=+2734.082826059" watchObservedRunningTime="2026-07-18 01:25:54.557001202 +0000 UTC m=+2734.084506161"
Jul 18 01:25:55.548702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:55.548670    2567 generic.go:356] "Generic (PLEG): container finished" podID="642992f9-8bcc-471f-bca6-d620a683d03b" containerID="ba34079bc6b67c2dcba5bc0a990f9c8eb32730408874472189232d1b2bcdedc4" exitCode=6
Jul 18 01:25:55.548702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:55.548707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" event={"ID":"642992f9-8bcc-471f-bca6-d620a683d03b","Type":"ContainerDied","Data":"ba34079bc6b67c2dcba5bc0a990f9c8eb32730408874472189232d1b2bcdedc4"}
Jul 18 01:25:55.549200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:55.548931    2567 scope.go:117] "RemoveContainer" containerID="ba34079bc6b67c2dcba5bc0a990f9c8eb32730408874472189232d1b2bcdedc4"
Jul 18 01:25:56.556607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:56.556576    2567 generic.go:356] "Generic (PLEG): container finished" podID="642992f9-8bcc-471f-bca6-d620a683d03b" containerID="34f8ca1a59c57023ab7846922fd44d88161cb7fbeaa422782ee2277a4aad3922" exitCode=6
Jul 18 01:25:56.557057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:56.556635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" event={"ID":"642992f9-8bcc-471f-bca6-d620a683d03b","Type":"ContainerDied","Data":"34f8ca1a59c57023ab7846922fd44d88161cb7fbeaa422782ee2277a4aad3922"}
Jul 18 01:25:56.557057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:56.556683    2567 scope.go:117] "RemoveContainer" containerID="ba34079bc6b67c2dcba5bc0a990f9c8eb32730408874472189232d1b2bcdedc4"
Jul 18 01:25:56.557057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:56.556965    2567 scope.go:117] "RemoveContainer" containerID="34f8ca1a59c57023ab7846922fd44d88161cb7fbeaa422782ee2277a4aad3922"
Jul 18 01:25:56.557243 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:56.557213    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-45wxj_odh-ai-gateway-infra(642992f9-8bcc-471f-bca6-d620a683d03b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" podUID="642992f9-8bcc-471f-bca6-d620a683d03b"
Jul 18 01:25:57.569786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:25:57.569748    2567 scope.go:117] "RemoveContainer" containerID="34f8ca1a59c57023ab7846922fd44d88161cb7fbeaa422782ee2277a4aad3922"
Jul 18 01:25:57.570205 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:25:57.569942    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-45wxj_odh-ai-gateway-infra(642992f9-8bcc-471f-bca6-d620a683d03b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" podUID="642992f9-8bcc-471f-bca6-d620a683d03b"
Jul 18 01:26:01.170748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.170708    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"]
Jul 18 01:26:01.174235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.174216    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:01.180804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.180764    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"]
Jul 18 01:26:01.302098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.302034    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l\" (UID: \"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:01.402955 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.402903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l\" (UID: \"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:01.403588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.403566    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l\" (UID: \"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:01.487555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.487465    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:01.638227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:01.638203    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"]
Jul 18 01:26:01.640722 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:26:01.640694    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab8ade6_49a2_4b19_8a6d_6e041c95dae8.slice/crio-4f1b580b3c8026140f1809ea7753d00980712f4177f983a2b4e2c3ccc6d1de2a WatchSource:0}: Error finding container 4f1b580b3c8026140f1809ea7753d00980712f4177f983a2b4e2c3ccc6d1de2a: Status 404 returned error can't find the container with id 4f1b580b3c8026140f1809ea7753d00980712f4177f983a2b4e2c3ccc6d1de2a
Jul 18 01:26:02.599303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:02.599249    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerID="7452a5e7e874253bc44cfdcd900d896c42eb50961c9f1852e1783c087481535e" exitCode=6
Jul 18 01:26:02.599719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:02.599326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" event={"ID":"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8","Type":"ContainerDied","Data":"7452a5e7e874253bc44cfdcd900d896c42eb50961c9f1852e1783c087481535e"}
Jul 18 01:26:02.599719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:02.599367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" event={"ID":"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8","Type":"ContainerStarted","Data":"4f1b580b3c8026140f1809ea7753d00980712f4177f983a2b4e2c3ccc6d1de2a"}
Jul 18 01:26:02.599719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:02.599556    2567 scope.go:117] "RemoveContainer" containerID="7452a5e7e874253bc44cfdcd900d896c42eb50961c9f1852e1783c087481535e"
Jul 18 01:26:03.606991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:03.606955    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerID="adde17adb119bf20a0714c73705460e0a04499f66f4df724354af457368e1238" exitCode=6
Jul 18 01:26:03.607505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:03.607038    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" event={"ID":"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8","Type":"ContainerDied","Data":"adde17adb119bf20a0714c73705460e0a04499f66f4df724354af457368e1238"}
Jul 18 01:26:03.607505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:03.607140    2567 scope.go:117] "RemoveContainer" containerID="7452a5e7e874253bc44cfdcd900d896c42eb50961c9f1852e1783c087481535e"
Jul 18 01:26:03.607505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:03.607354    2567 scope.go:117] "RemoveContainer" containerID="adde17adb119bf20a0714c73705460e0a04499f66f4df724354af457368e1238"
Jul 18 01:26:03.607665 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:03.607645    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l_odh-ai-gateway-infra(5ab8ade6-49a2-4b19-8a6d-6e041c95dae8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8"
Jul 18 01:26:03.913351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:03.913238    2567 scope.go:117] "RemoveContainer" containerID="e4bae6c08b3f6e29b7a41524dd52b02bb9042f5c1bde9af44226f0d5dd57dcf1"
Jul 18 01:26:04.615938 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:04.615906    2567 generic.go:356] "Generic (PLEG): container finished" podID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerID="c15bc5ec1cd1e3c994d2e879a1f05de854ec3c9de6fe01a4c5eb4574e621e52a" exitCode=6
Jul 18 01:26:04.616386 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:04.615991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" event={"ID":"90f49001-9106-427e-a6c1-05fbcb58eb5f","Type":"ContainerDied","Data":"c15bc5ec1cd1e3c994d2e879a1f05de854ec3c9de6fe01a4c5eb4574e621e52a"}
Jul 18 01:26:04.616386 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:04.616070    2567 scope.go:117] "RemoveContainer" containerID="e4bae6c08b3f6e29b7a41524dd52b02bb9042f5c1bde9af44226f0d5dd57dcf1"
Jul 18 01:26:04.616386 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:04.616386    2567 scope.go:117] "RemoveContainer" containerID="c15bc5ec1cd1e3c994d2e879a1f05de854ec3c9de6fe01a4c5eb4574e621e52a"
Jul 18 01:26:04.616649 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:04.616612    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84_odh-ai-gateway-infra(90f49001-9106-427e-a6c1-05fbcb58eb5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f"
Jul 18 01:26:04.617957 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:04.617938    2567 scope.go:117] "RemoveContainer" containerID="adde17adb119bf20a0714c73705460e0a04499f66f4df724354af457368e1238"
Jul 18 01:26:04.618181 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:04.618161    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l_odh-ai-gateway-infra(5ab8ade6-49a2-4b19-8a6d-6e041c95dae8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8"
Jul 18 01:26:05.641199 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:05.641157    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"]
Jul 18 01:26:05.791525 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:05.791492    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:26:05.945042 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:05.944953    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f49001-9106-427e-a6c1-05fbcb58eb5f-maas-api-service-ca\") pod \"90f49001-9106-427e-a6c1-05fbcb58eb5f\" (UID: \"90f49001-9106-427e-a6c1-05fbcb58eb5f\") "
Jul 18 01:26:05.945447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:05.945419    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f49001-9106-427e-a6c1-05fbcb58eb5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90f49001-9106-427e-a6c1-05fbcb58eb5f" (UID: "90f49001-9106-427e-a6c1-05fbcb58eb5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:26:06.046535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.046493    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90f49001-9106-427e-a6c1-05fbcb58eb5f-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:06.200350 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200250    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"]
Jul 18 01:26:06.200669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200656    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200671    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200678    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200684    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200696    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200702    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200857 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200760    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.200857 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.200772    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:06.203748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.203729    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:06.211659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.211633    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"]
Jul 18 01:26:06.349753 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.349718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/930b8e15-74f9-45d9-85f4-d43e6ee0642a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc\" (UID: \"930b8e15-74f9-45d9-85f4-d43e6ee0642a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:06.450659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.450566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/930b8e15-74f9-45d9-85f4-d43e6ee0642a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc\" (UID: \"930b8e15-74f9-45d9-85f4-d43e6ee0642a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:06.451219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.451200    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/930b8e15-74f9-45d9-85f4-d43e6ee0642a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc\" (UID: \"930b8e15-74f9-45d9-85f4-d43e6ee0642a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:06.516419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.516376    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:06.632747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.632710    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84" event={"ID":"90f49001-9106-427e-a6c1-05fbcb58eb5f","Type":"ContainerDied","Data":"7527c4ae450dd60945d392a5b88aad326d01bef88b868f6708081c83ff22ec7e"}
Jul 18 01:26:06.632956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.632766    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"
Jul 18 01:26:06.632956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.632788    2567 scope.go:117] "RemoveContainer" containerID="c15bc5ec1cd1e3c994d2e879a1f05de854ec3c9de6fe01a4c5eb4574e621e52a"
Jul 18 01:26:06.661562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.661488    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"]
Jul 18 01:26:06.663571 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:26:06.663545    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930b8e15_74f9_45d9_85f4_d43e6ee0642a.slice/crio-4ba5f01e5f8ed9dab60e2f96965b9ab818eb098aa7853f0eeabc7762b4040446 WatchSource:0}: Error finding container 4ba5f01e5f8ed9dab60e2f96965b9ab818eb098aa7853f0eeabc7762b4040446: Status 404 returned error can't find the container with id 4ba5f01e5f8ed9dab60e2f96965b9ab818eb098aa7853f0eeabc7762b4040446
Jul 18 01:26:06.664928 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.664901    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"]
Jul 18 01:26:06.667915 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.667896    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-n5m84"]
Jul 18 01:26:06.928575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:06.928539    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" path="/var/lib/kubelet/pods/90f49001-9106-427e-a6c1-05fbcb58eb5f/volumes"
Jul 18 01:26:07.640496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:07.640411    2567 generic.go:356] "Generic (PLEG): container finished" podID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerID="6e38a51b73549aaa68e724a339a54579084a5631511b68594ebfef867744cf7c" exitCode=6
Jul 18 01:26:07.640496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:07.640473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" event={"ID":"930b8e15-74f9-45d9-85f4-d43e6ee0642a","Type":"ContainerDied","Data":"6e38a51b73549aaa68e724a339a54579084a5631511b68594ebfef867744cf7c"}
Jul 18 01:26:07.640684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:07.640513    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" event={"ID":"930b8e15-74f9-45d9-85f4-d43e6ee0642a","Type":"ContainerStarted","Data":"4ba5f01e5f8ed9dab60e2f96965b9ab818eb098aa7853f0eeabc7762b4040446"}
Jul 18 01:26:07.640684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:07.640663    2567 scope.go:117] "RemoveContainer" containerID="6e38a51b73549aaa68e724a339a54579084a5631511b68594ebfef867744cf7c"
Jul 18 01:26:07.777225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:07.777200    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:07.879781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:07.879747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:08.647980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:08.647892    2567 generic.go:356] "Generic (PLEG): container finished" podID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerID="a6f83e6571db1dae447965d8134dc7ef275accf9d2e7d59007fc52670fcc0e81" exitCode=6
Jul 18 01:26:08.648122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:08.647982    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" event={"ID":"930b8e15-74f9-45d9-85f4-d43e6ee0642a","Type":"ContainerDied","Data":"a6f83e6571db1dae447965d8134dc7ef275accf9d2e7d59007fc52670fcc0e81"}
Jul 18 01:26:08.648122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:08.648041    2567 scope.go:117] "RemoveContainer" containerID="6e38a51b73549aaa68e724a339a54579084a5631511b68594ebfef867744cf7c"
Jul 18 01:26:08.648206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:08.648190    2567 scope.go:117] "RemoveContainer" containerID="a6f83e6571db1dae447965d8134dc7ef275accf9d2e7d59007fc52670fcc0e81"
Jul 18 01:26:08.648477 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:08.648457    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc_odh-ai-gateway-infra(930b8e15-74f9-45d9-85f4-d43e6ee0642a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a"
Jul 18 01:26:08.977900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:08.977782    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:09.655254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:09.655225    2567 scope.go:117] "RemoveContainer" containerID="a6f83e6571db1dae447965d8134dc7ef275accf9d2e7d59007fc52670fcc0e81"
Jul 18 01:26:09.655530 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:09.655469    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc_odh-ai-gateway-infra(930b8e15-74f9-45d9-85f4-d43e6ee0642a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a"
Jul 18 01:26:11.913613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:11.913577    2567 scope.go:117] "RemoveContainer" containerID="34f8ca1a59c57023ab7846922fd44d88161cb7fbeaa422782ee2277a4aad3922"
Jul 18 01:26:12.672952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:12.672914    2567 generic.go:356] "Generic (PLEG): container finished" podID="642992f9-8bcc-471f-bca6-d620a683d03b" containerID="edb16f513ef02f0886d64cc53f250cb74c4d41e196c6293d6238f785460c6073" exitCode=6
Jul 18 01:26:12.673125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:12.672988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" event={"ID":"642992f9-8bcc-471f-bca6-d620a683d03b","Type":"ContainerDied","Data":"edb16f513ef02f0886d64cc53f250cb74c4d41e196c6293d6238f785460c6073"}
Jul 18 01:26:12.673125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:12.673083    2567 scope.go:117] "RemoveContainer" containerID="34f8ca1a59c57023ab7846922fd44d88161cb7fbeaa422782ee2277a4aad3922"
Jul 18 01:26:12.673462 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:12.673438    2567 scope.go:117] "RemoveContainer" containerID="edb16f513ef02f0886d64cc53f250cb74c4d41e196c6293d6238f785460c6073"
Jul 18 01:26:12.673679 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:12.673651    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-45wxj_odh-ai-gateway-infra(642992f9-8bcc-471f-bca6-d620a683d03b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" podUID="642992f9-8bcc-471f-bca6-d620a683d03b"
Jul 18 01:26:13.699758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:13.699721    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"]
Jul 18 01:26:13.841604 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:13.841581    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:26:14.016593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.016505    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/642992f9-8bcc-471f-bca6-d620a683d03b-maas-api-service-ca\") pod \"642992f9-8bcc-471f-bca6-d620a683d03b\" (UID: \"642992f9-8bcc-471f-bca6-d620a683d03b\") "
Jul 18 01:26:14.016870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.016843    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/642992f9-8bcc-471f-bca6-d620a683d03b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "642992f9-8bcc-471f-bca6-d620a683d03b" (UID: "642992f9-8bcc-471f-bca6-d620a683d03b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:26:14.117254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.117215    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/642992f9-8bcc-471f-bca6-d620a683d03b-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:14.686816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.686776    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj" event={"ID":"642992f9-8bcc-471f-bca6-d620a683d03b","Type":"ContainerDied","Data":"352034cdefc94b93c514ade5c346058a1c28a4013cd9df7a8acb42813d6de920"}
Jul 18 01:26:14.686816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.686799    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"
Jul 18 01:26:14.687091 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.686841    2567 scope.go:117] "RemoveContainer" containerID="edb16f513ef02f0886d64cc53f250cb74c4d41e196c6293d6238f785460c6073"
Jul 18 01:26:14.706826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.706786    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"]
Jul 18 01:26:14.716178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.716148    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-45wxj"]
Jul 18 01:26:14.918764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:14.918732    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" path="/var/lib/kubelet/pods/642992f9-8bcc-471f-bca6-d620a683d03b/volumes"
Jul 18 01:26:17.913647 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:17.913610    2567 scope.go:117] "RemoveContainer" containerID="adde17adb119bf20a0714c73705460e0a04499f66f4df724354af457368e1238"
Jul 18 01:26:18.710520 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:18.710487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" event={"ID":"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8","Type":"ContainerStarted","Data":"6293f72f4c16073855e7ec0a2cc3714f7011e273f298aecdd03c68669a3af7bd"}
Jul 18 01:26:18.729423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:18.729363    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" podStartSLOduration=17.729346419 podStartE2EDuration="17.729346419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:26:18.727804678 +0000 UTC m=+2758.255309640" watchObservedRunningTime="2026-07-18 01:26:18.729346419 +0000 UTC m=+2758.256851380"
Jul 18 01:26:19.717256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:19.717223    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerID="6293f72f4c16073855e7ec0a2cc3714f7011e273f298aecdd03c68669a3af7bd" exitCode=6
Jul 18 01:26:19.717706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:19.717297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" event={"ID":"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8","Type":"ContainerDied","Data":"6293f72f4c16073855e7ec0a2cc3714f7011e273f298aecdd03c68669a3af7bd"}
Jul 18 01:26:19.717706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:19.717363    2567 scope.go:117] "RemoveContainer" containerID="adde17adb119bf20a0714c73705460e0a04499f66f4df724354af457368e1238"
Jul 18 01:26:19.717706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:19.717644    2567 scope.go:117] "RemoveContainer" containerID="6293f72f4c16073855e7ec0a2cc3714f7011e273f298aecdd03c68669a3af7bd"
Jul 18 01:26:19.717884 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:19.717857    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l_odh-ai-gateway-infra(5ab8ade6-49a2-4b19-8a6d-6e041c95dae8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8"
Jul 18 01:26:19.737704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:19.737672    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"]
Jul 18 01:26:20.867210 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:20.867182    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:20.915701 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:20.915669    2567 scope.go:117] "RemoveContainer" containerID="a6f83e6571db1dae447965d8134dc7ef275accf9d2e7d59007fc52670fcc0e81"
Jul 18 01:26:20.971447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:20.971412    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8-maas-api-service-ca\") pod \"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8\" (UID: \"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8\") "
Jul 18 01:26:20.971808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:20.971766    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" (UID: "5ab8ade6-49a2-4b19-8a6d-6e041c95dae8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:26:21.072717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.072678    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:21.737406 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.737309    2567 generic.go:356] "Generic (PLEG): container finished" podID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerID="04bbfe72e8fab7c3072b632020d614bc10ddc08ea8c75848ecf07251c54a4964" exitCode=6
Jul 18 01:26:21.737406 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.737382    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" event={"ID":"930b8e15-74f9-45d9-85f4-d43e6ee0642a","Type":"ContainerDied","Data":"04bbfe72e8fab7c3072b632020d614bc10ddc08ea8c75848ecf07251c54a4964"}
Jul 18 01:26:21.737645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.737479    2567 scope.go:117] "RemoveContainer" containerID="a6f83e6571db1dae447965d8134dc7ef275accf9d2e7d59007fc52670fcc0e81"
Jul 18 01:26:21.737813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.737795    2567 scope.go:117] "RemoveContainer" containerID="04bbfe72e8fab7c3072b632020d614bc10ddc08ea8c75848ecf07251c54a4964"
Jul 18 01:26:21.738056 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:21.738029    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc_odh-ai-gateway-infra(930b8e15-74f9-45d9-85f4-d43e6ee0642a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a"
Jul 18 01:26:21.739151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.738857    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l" event={"ID":"5ab8ade6-49a2-4b19-8a6d-6e041c95dae8","Type":"ContainerDied","Data":"4f1b580b3c8026140f1809ea7753d00980712f4177f983a2b4e2c3ccc6d1de2a"}
Jul 18 01:26:21.739151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.738872    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"
Jul 18 01:26:21.750251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.750229    2567 scope.go:117] "RemoveContainer" containerID="6293f72f4c16073855e7ec0a2cc3714f7011e273f298aecdd03c68669a3af7bd"
Jul 18 01:26:21.769831 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.769798    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"]
Jul 18 01:26:21.773538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:21.773511    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-bp64l"]
Jul 18 01:26:22.769883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:22.769848    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"]
Jul 18 01:26:22.913798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:22.913768    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:22.918735 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:22.918701    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" path="/var/lib/kubelet/pods/5ab8ade6-49a2-4b19-8a6d-6e041c95dae8/volumes"
Jul 18 01:26:22.989768 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:22.989731    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/930b8e15-74f9-45d9-85f4-d43e6ee0642a-maas-api-service-ca\") pod \"930b8e15-74f9-45d9-85f4-d43e6ee0642a\" (UID: \"930b8e15-74f9-45d9-85f4-d43e6ee0642a\") "
Jul 18 01:26:22.990187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:22.990160    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/930b8e15-74f9-45d9-85f4-d43e6ee0642a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "930b8e15-74f9-45d9-85f4-d43e6ee0642a" (UID: "930b8e15-74f9-45d9-85f4-d43e6ee0642a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:26:23.091439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:23.091400    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/930b8e15-74f9-45d9-85f4-d43e6ee0642a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:23.751961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:23.751925    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"
Jul 18 01:26:23.751961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:23.751955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc" event={"ID":"930b8e15-74f9-45d9-85f4-d43e6ee0642a","Type":"ContainerDied","Data":"4ba5f01e5f8ed9dab60e2f96965b9ab818eb098aa7853f0eeabc7762b4040446"}
Jul 18 01:26:23.752222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:23.752028    2567 scope.go:117] "RemoveContainer" containerID="04bbfe72e8fab7c3072b632020d614bc10ddc08ea8c75848ecf07251c54a4964"
Jul 18 01:26:23.772690 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:23.772662    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"]
Jul 18 01:26:23.776306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:23.776283    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-4qcbc"]
Jul 18 01:26:24.305364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:24.305332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:24.919431 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:24.919397    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" path="/var/lib/kubelet/pods/930b8e15-74f9-45d9-85f4-d43e6ee0642a/volumes"
Jul 18 01:26:24.933290 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:24.933234    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:30.522170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:30.522139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:35.959401 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:35.959361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:36.058212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:36.058180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:39.029985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.029944    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"]
Jul 18 01:26:39.030409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030388    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.030409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030401    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030412    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030418    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030426    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030432    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030445    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030450    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030458    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030463    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030470    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030475    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030485    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:39.030487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030490    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030559    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030569    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030574    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030579    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030585    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030592    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:39.031023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.030598    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90f49001-9106-427e-a6c1-05fbcb58eb5f" containerName="revoke-keys"
Jul 18 01:26:39.033835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.033817    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:39.036792 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.036774    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:26:39.036907 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.036831    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:26:39.040839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.040810    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"]
Jul 18 01:26:39.142853 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.142808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/026a80a8-a14e-4b74-821a-fd5ef6cfe545-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc\" (UID: \"026a80a8-a14e-4b74-821a-fd5ef6cfe545\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:39.244062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.244017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/026a80a8-a14e-4b74-821a-fd5ef6cfe545-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc\" (UID: \"026a80a8-a14e-4b74-821a-fd5ef6cfe545\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:39.244793 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.244692    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/026a80a8-a14e-4b74-821a-fd5ef6cfe545-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc\" (UID: \"026a80a8-a14e-4b74-821a-fd5ef6cfe545\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:39.346990 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.346948    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:39.485434 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.485410    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"]
Jul 18 01:26:39.487915 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:26:39.487884    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026a80a8_a14e_4b74_821a_fd5ef6cfe545.slice/crio-718a859f4dc45c972ff6fd7266adb0ce0aeaf0a2532256bf71462eea960079e7 WatchSource:0}: Error finding container 718a859f4dc45c972ff6fd7266adb0ce0aeaf0a2532256bf71462eea960079e7: Status 404 returned error can't find the container with id 718a859f4dc45c972ff6fd7266adb0ce0aeaf0a2532256bf71462eea960079e7
Jul 18 01:26:39.837648 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.837615    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" event={"ID":"026a80a8-a14e-4b74-821a-fd5ef6cfe545","Type":"ContainerStarted","Data":"37070507fe5004432e75c73a9716a145b9d86f632475fd8eab4de482490b0734"}
Jul 18 01:26:39.837872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.837669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" event={"ID":"026a80a8-a14e-4b74-821a-fd5ef6cfe545","Type":"ContainerStarted","Data":"718a859f4dc45c972ff6fd7266adb0ce0aeaf0a2532256bf71462eea960079e7"}
Jul 18 01:26:39.852377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:39.852321    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" podStartSLOduration=0.852308086 podStartE2EDuration="852.308086ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:26:39.850353278 +0000 UTC m=+2779.377858256" watchObservedRunningTime="2026-07-18 01:26:39.852308086 +0000 UTC m=+2779.379813045"
Jul 18 01:26:40.845548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:40.845511    2567 generic.go:356] "Generic (PLEG): container finished" podID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerID="37070507fe5004432e75c73a9716a145b9d86f632475fd8eab4de482490b0734" exitCode=6
Jul 18 01:26:40.846016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:40.845559    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" event={"ID":"026a80a8-a14e-4b74-821a-fd5ef6cfe545","Type":"ContainerDied","Data":"37070507fe5004432e75c73a9716a145b9d86f632475fd8eab4de482490b0734"}
Jul 18 01:26:40.846016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:40.845790    2567 scope.go:117] "RemoveContainer" containerID="37070507fe5004432e75c73a9716a145b9d86f632475fd8eab4de482490b0734"
Jul 18 01:26:41.851981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:41.851943    2567 generic.go:356] "Generic (PLEG): container finished" podID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerID="bceb4ae3cb4e7192d53e31a303e0680760028d51c85555a8734d5aafd15df2b8" exitCode=6
Jul 18 01:26:41.852454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:41.852043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" event={"ID":"026a80a8-a14e-4b74-821a-fd5ef6cfe545","Type":"ContainerDied","Data":"bceb4ae3cb4e7192d53e31a303e0680760028d51c85555a8734d5aafd15df2b8"}
Jul 18 01:26:41.852454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:41.852104    2567 scope.go:117] "RemoveContainer" containerID="37070507fe5004432e75c73a9716a145b9d86f632475fd8eab4de482490b0734"
Jul 18 01:26:41.852454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:41.852327    2567 scope.go:117] "RemoveContainer" containerID="bceb4ae3cb4e7192d53e31a303e0680760028d51c85555a8734d5aafd15df2b8"
Jul 18 01:26:41.852703 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:41.852600    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc_odh-ai-gateway-infra(026a80a8-a14e-4b74-821a-fd5ef6cfe545)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545"
Jul 18 01:26:42.858709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:42.858680    2567 scope.go:117] "RemoveContainer" containerID="bceb4ae3cb4e7192d53e31a303e0680760028d51c85555a8734d5aafd15df2b8"
Jul 18 01:26:42.859115 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:42.858867    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc_odh-ai-gateway-infra(026a80a8-a14e-4b74-821a-fd5ef6cfe545)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545"
Jul 18 01:26:44.974396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:44.974361    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"]
Jul 18 01:26:44.974821 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:44.974569    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy" containerID="cri-o://1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969" gracePeriod=30
Jul 18 01:26:44.993577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:44.993543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:45.241212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:45.241120    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:46.246659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:46.246616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:46.346731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:46.346691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:46.446335 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:46.446262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:47.120485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:47.120453    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:47.224088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:47.224026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:47.325013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:47.324969    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:48.653502 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:48.653469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:49.117912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:49.117875    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:26:50.243849 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.243787    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:26:50.346305 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346220    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-data\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346305 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346261    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6748\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-kube-api-access-v6748\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346322    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-certs\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346341    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc9b192-e303-4938-935a-97229dede9bf-istio-podinfo\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346372    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-credential-socket\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346389    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-envoy\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346414    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9b192-e303-4938-935a-97229dede9bf-istiod-ca-cert\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346486    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-socket\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346516    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-istio-token\") pod \"5cc9b192-e303-4938-935a-97229dede9bf\" (UID: \"5cc9b192-e303-4938-935a-97229dede9bf\") "
Jul 18 01:26:50.346641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346540    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:26:50.347160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346697    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:26:50.347160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346784    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cc9b192-e303-4938-935a-97229dede9bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:26:50.347160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346808    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.347160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346827    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.347160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.346979    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:26:50.347465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.347194    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:26:50.349333 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.349300    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cc9b192-e303-4938-935a-97229dede9bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:26:50.349454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.349411    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:26:50.349454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.349439    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:26:50.349567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.349490    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-kube-api-access-v6748" (OuterVolumeSpecName: "kube-api-access-v6748") pod "5cc9b192-e303-4938-935a-97229dede9bf" (UID: "5cc9b192-e303-4938-935a-97229dede9bf"). InnerVolumeSpecName "kube-api-access-v6748". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:26:50.448541 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448493    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.448541 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448543    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.448825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448558    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.448825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448573    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6748\" (UniqueName: \"kubernetes.io/projected/5cc9b192-e303-4938-935a-97229dede9bf-kube-api-access-v6748\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.448825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448587    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc9b192-e303-4938-935a-97229dede9bf-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.448825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448600    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc9b192-e303-4938-935a-97229dede9bf-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.448825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.448612    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9b192-e303-4938-935a-97229dede9bf-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:50.907152 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.907117    2567 generic.go:356] "Generic (PLEG): container finished" podID="5cc9b192-e303-4938-935a-97229dede9bf" containerID="1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969" exitCode=0
Jul 18 01:26:50.907396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.907186    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"
Jul 18 01:26:50.907396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.907202    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" event={"ID":"5cc9b192-e303-4938-935a-97229dede9bf","Type":"ContainerDied","Data":"1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969"}
Jul 18 01:26:50.907396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.907291    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57" event={"ID":"5cc9b192-e303-4938-935a-97229dede9bf","Type":"ContainerDied","Data":"7c79bd4178892c765b5383cc1d49c46041a498b17d8c7af55023b362ae7dbbe0"}
Jul 18 01:26:50.907396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.907317    2567 scope.go:117] "RemoveContainer" containerID="1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969"
Jul 18 01:26:50.919282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.919249    2567 scope.go:117] "RemoveContainer" containerID="4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"
Jul 18 01:26:50.930392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.930368    2567 scope.go:117] "RemoveContainer" containerID="1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969"
Jul 18 01:26:50.930716 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:50.930695    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969\": container with ID starting with 1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969 not found: ID does not exist" containerID="1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969"
Jul 18 01:26:50.930786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.930723    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969"} err="failed to get container status \"1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969\": rpc error: code = NotFound desc = could not find container \"1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969\": container with ID starting with 1ab82759235c28d5cbd77e96e59d64f1c485e7160c7e0d83e3ec25da58607969 not found: ID does not exist"
Jul 18 01:26:50.930786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.930741    2567 scope.go:117] "RemoveContainer" containerID="4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"
Jul 18 01:26:50.931008 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:50.930986    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13\": container with ID starting with 4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13 not found: ID does not exist" containerID="4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"
Jul 18 01:26:50.931053 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.931021    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13"} err="failed to get container status \"4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13\": rpc error: code = NotFound desc = could not find container \"4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13\": container with ID starting with 4020b2c0fb4544937eeffd05eca60f019d38870806e7b99221289e90cdb1ea13 not found: ID does not exist"
Jul 18 01:26:50.931233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.931218    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"]
Jul 18 01:26:50.934495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:50.934467    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f92ba3bc-openshift-default-78765fdcff-mhs57"]
Jul 18 01:26:52.924482 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:52.921048    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cc9b192-e303-4938-935a-97229dede9bf" path="/var/lib/kubelet/pods/5cc9b192-e303-4938-935a-97229dede9bf/volumes"
Jul 18 01:26:53.646971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.646937    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"]
Jul 18 01:26:53.647367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647354    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy"
Jul 18 01:26:53.647419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647368    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy"
Jul 18 01:26:53.647419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647384    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:53.647419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647390    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:53.647419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647402    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy"
Jul 18 01:26:53.647419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647407    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy"
Jul 18 01:26:53.647419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647419    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:53.647602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647424    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:53.647602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647482    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="642992f9-8bcc-471f-bca6-d620a683d03b" containerName="revoke-keys"
Jul 18 01:26:53.647602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647490    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy"
Jul 18 01:26:53.647602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647499    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ab8ade6-49a2-4b19-8a6d-6e041c95dae8" containerName="revoke-keys"
Jul 18 01:26:53.647602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.647506    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="930b8e15-74f9-45d9-85f4-d43e6ee0642a" containerName="revoke-keys"
Jul 18 01:26:53.652107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.652088    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:26:53.657750 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.657615    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"]
Jul 18 01:26:53.777320 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.777253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aff11209-7dcf-49e6-bc60-49d8353c88f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-jcb76\" (UID: \"aff11209-7dcf-49e6-bc60-49d8353c88f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:26:53.878388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.878348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aff11209-7dcf-49e6-bc60-49d8353c88f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-jcb76\" (UID: \"aff11209-7dcf-49e6-bc60-49d8353c88f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:26:53.878942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.878923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aff11209-7dcf-49e6-bc60-49d8353c88f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-jcb76\" (UID: \"aff11209-7dcf-49e6-bc60-49d8353c88f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:26:53.964727 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:53.964636    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:26:54.105471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.105438    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"]
Jul 18 01:26:54.108763 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:26:54.108739    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff11209_7dcf_49e6_bc60_49d8353c88f5.slice/crio-20902cb8e63cc43aae895f00c276bd8f91c3076b148c3b38ebf4186ab5110102 WatchSource:0}: Error finding container 20902cb8e63cc43aae895f00c276bd8f91c3076b148c3b38ebf4186ab5110102: Status 404 returned error can't find the container with id 20902cb8e63cc43aae895f00c276bd8f91c3076b148c3b38ebf4186ab5110102
Jul 18 01:26:54.622437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.622397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:54.722185 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.722095    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:54.913060 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.913018    2567 scope.go:117] "RemoveContainer" containerID="bceb4ae3cb4e7192d53e31a303e0680760028d51c85555a8734d5aafd15df2b8"
Jul 18 01:26:54.930568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.930527    2567 generic.go:356] "Generic (PLEG): container finished" podID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerID="def45d302375390d2dea689a1c7a41a032a02061b46026f5fb5317aab31c0afb" exitCode=6
Jul 18 01:26:54.930701 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.930603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" event={"ID":"aff11209-7dcf-49e6-bc60-49d8353c88f5","Type":"ContainerDied","Data":"def45d302375390d2dea689a1c7a41a032a02061b46026f5fb5317aab31c0afb"}
Jul 18 01:26:54.930701 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.930669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" event={"ID":"aff11209-7dcf-49e6-bc60-49d8353c88f5","Type":"ContainerStarted","Data":"20902cb8e63cc43aae895f00c276bd8f91c3076b148c3b38ebf4186ab5110102"}
Jul 18 01:26:54.930902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:54.930887    2567 scope.go:117] "RemoveContainer" containerID="def45d302375390d2dea689a1c7a41a032a02061b46026f5fb5317aab31c0afb"
Jul 18 01:26:55.268087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.268055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:26:55.937746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.937713    2567 generic.go:356] "Generic (PLEG): container finished" podID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerID="2b1dcbc466259b0081914c01d568d2a54f6625aa12d346acba3df5cb6c3c01ea" exitCode=6
Jul 18 01:26:55.937940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.937795    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" event={"ID":"026a80a8-a14e-4b74-821a-fd5ef6cfe545","Type":"ContainerDied","Data":"2b1dcbc466259b0081914c01d568d2a54f6625aa12d346acba3df5cb6c3c01ea"}
Jul 18 01:26:55.937940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.937852    2567 scope.go:117] "RemoveContainer" containerID="bceb4ae3cb4e7192d53e31a303e0680760028d51c85555a8734d5aafd15df2b8"
Jul 18 01:26:55.938214 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.938188    2567 scope.go:117] "RemoveContainer" containerID="2b1dcbc466259b0081914c01d568d2a54f6625aa12d346acba3df5cb6c3c01ea"
Jul 18 01:26:55.938462 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:55.938442    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc_odh-ai-gateway-infra(026a80a8-a14e-4b74-821a-fd5ef6cfe545)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545"
Jul 18 01:26:55.939710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.939689    2567 generic.go:356] "Generic (PLEG): container finished" podID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerID="b91cfab9651d9d88f0dcafe702e4f56cbbe2e3328d3d5094bf309104379ede7d" exitCode=6
Jul 18 01:26:55.939811 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.939780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" event={"ID":"aff11209-7dcf-49e6-bc60-49d8353c88f5","Type":"ContainerDied","Data":"b91cfab9651d9d88f0dcafe702e4f56cbbe2e3328d3d5094bf309104379ede7d"}
Jul 18 01:26:55.939935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.939923    2567 scope.go:117] "RemoveContainer" containerID="b91cfab9651d9d88f0dcafe702e4f56cbbe2e3328d3d5094bf309104379ede7d"
Jul 18 01:26:55.940111 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:55.940093    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-jcb76_odh-ai-gateway-infra(aff11209-7dcf-49e6-bc60-49d8353c88f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5"
Jul 18 01:26:55.949539 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:55.949506    2567 scope.go:117] "RemoveContainer" containerID="def45d302375390d2dea689a1c7a41a032a02061b46026f5fb5317aab31c0afb"
Jul 18 01:26:56.243623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.243540    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"]
Jul 18 01:26:56.244006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.243992    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cc9b192-e303-4938-935a-97229dede9bf" containerName="istio-proxy"
Jul 18 01:26:56.248175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.248158    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:26:56.254949 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.254143    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"]
Jul 18 01:26:56.403297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.403232    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa4d608-50d5-4892-adcd-2432188166f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf\" (UID: \"6aa4d608-50d5-4892-adcd-2432188166f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:26:56.503981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.503883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa4d608-50d5-4892-adcd-2432188166f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf\" (UID: \"6aa4d608-50d5-4892-adcd-2432188166f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:26:56.504519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.504496    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa4d608-50d5-4892-adcd-2432188166f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf\" (UID: \"6aa4d608-50d5-4892-adcd-2432188166f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:26:56.562454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.562418    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:26:56.707891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.707864    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"]
Jul 18 01:26:56.710618 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:26:56.710571    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa4d608_50d5_4892_adcd_2432188166f4.slice/crio-da38bceef83f1f7b3b084f15b48196a5592f6cedeff3cc37e2dc7d750548cf3d WatchSource:0}: Error finding container da38bceef83f1f7b3b084f15b48196a5592f6cedeff3cc37e2dc7d750548cf3d: Status 404 returned error can't find the container with id da38bceef83f1f7b3b084f15b48196a5592f6cedeff3cc37e2dc7d750548cf3d
Jul 18 01:26:56.946534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.946501    2567 scope.go:117] "RemoveContainer" containerID="b91cfab9651d9d88f0dcafe702e4f56cbbe2e3328d3d5094bf309104379ede7d"
Jul 18 01:26:56.946797 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:56.946767    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-jcb76_odh-ai-gateway-infra(aff11209-7dcf-49e6-bc60-49d8353c88f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5"
Jul 18 01:26:56.947878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.947853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" event={"ID":"6aa4d608-50d5-4892-adcd-2432188166f4","Type":"ContainerStarted","Data":"ce63b16734d1e3172aee2cba8edeffc0b708474549614eafde444f357156c239"}
Jul 18 01:26:56.947994 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.947897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" event={"ID":"6aa4d608-50d5-4892-adcd-2432188166f4","Type":"ContainerStarted","Data":"da38bceef83f1f7b3b084f15b48196a5592f6cedeff3cc37e2dc7d750548cf3d"}
Jul 18 01:26:56.965109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.965072    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"]
Jul 18 01:26:56.976834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:56.976777    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" podStartSLOduration=0.976759173 podStartE2EDuration="976.759173ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:26:56.974334094 +0000 UTC m=+2796.501839048" watchObservedRunningTime="2026-07-18 01:26:56.976759173 +0000 UTC m=+2796.504264133"
Jul 18 01:26:57.100389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.100358    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:57.210582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.210544    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/026a80a8-a14e-4b74-821a-fd5ef6cfe545-maas-api-service-ca\") pod \"026a80a8-a14e-4b74-821a-fd5ef6cfe545\" (UID: \"026a80a8-a14e-4b74-821a-fd5ef6cfe545\") "
Jul 18 01:26:57.210948 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.210922    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026a80a8-a14e-4b74-821a-fd5ef6cfe545-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "026a80a8-a14e-4b74-821a-fd5ef6cfe545" (UID: "026a80a8-a14e-4b74-821a-fd5ef6cfe545"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:26:57.311758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.311722    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/026a80a8-a14e-4b74-821a-fd5ef6cfe545-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:26:57.960000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.959909    2567 generic.go:356] "Generic (PLEG): container finished" podID="6aa4d608-50d5-4892-adcd-2432188166f4" containerID="ce63b16734d1e3172aee2cba8edeffc0b708474549614eafde444f357156c239" exitCode=6
Jul 18 01:26:57.960439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.960003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" event={"ID":"6aa4d608-50d5-4892-adcd-2432188166f4","Type":"ContainerDied","Data":"ce63b16734d1e3172aee2cba8edeffc0b708474549614eafde444f357156c239"}
Jul 18 01:26:57.960439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.960199    2567 scope.go:117] "RemoveContainer" containerID="ce63b16734d1e3172aee2cba8edeffc0b708474549614eafde444f357156c239"
Jul 18 01:26:57.961557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.961533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc" event={"ID":"026a80a8-a14e-4b74-821a-fd5ef6cfe545","Type":"ContainerDied","Data":"718a859f4dc45c972ff6fd7266adb0ce0aeaf0a2532256bf71462eea960079e7"}
Jul 18 01:26:57.961660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.961558    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"
Jul 18 01:26:57.961660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.961583    2567 scope.go:117] "RemoveContainer" containerID="2b1dcbc466259b0081914c01d568d2a54f6625aa12d346acba3df5cb6c3c01ea"
Jul 18 01:26:57.993082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.993053    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"]
Jul 18 01:26:57.996548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:57.996524    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ks9cc"]
Jul 18 01:26:58.919523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:58.919490    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" path="/var/lib/kubelet/pods/026a80a8-a14e-4b74-821a-fd5ef6cfe545/volumes"
Jul 18 01:26:58.969883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:58.969847    2567 generic.go:356] "Generic (PLEG): container finished" podID="6aa4d608-50d5-4892-adcd-2432188166f4" containerID="6715402eec8b74f6dec3e5328a8a68edbb0e2a1ac07c56e58d8687f435f33ea4" exitCode=6
Jul 18 01:26:58.970330 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:58.969911    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" event={"ID":"6aa4d608-50d5-4892-adcd-2432188166f4","Type":"ContainerDied","Data":"6715402eec8b74f6dec3e5328a8a68edbb0e2a1ac07c56e58d8687f435f33ea4"}
Jul 18 01:26:58.970330 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:58.969972    2567 scope.go:117] "RemoveContainer" containerID="ce63b16734d1e3172aee2cba8edeffc0b708474549614eafde444f357156c239"
Jul 18 01:26:58.970330 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:58.970195    2567 scope.go:117] "RemoveContainer" containerID="6715402eec8b74f6dec3e5328a8a68edbb0e2a1ac07c56e58d8687f435f33ea4"
Jul 18 01:26:58.970479 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:58.970404    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf_odh-ai-gateway-infra(6aa4d608-50d5-4892-adcd-2432188166f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" podUID="6aa4d608-50d5-4892-adcd-2432188166f4"
Jul 18 01:26:59.976705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:26:59.976678    2567 scope.go:117] "RemoveContainer" containerID="6715402eec8b74f6dec3e5328a8a68edbb0e2a1ac07c56e58d8687f435f33ea4"
Jul 18 01:26:59.977201 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:26:59.976944    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf_odh-ai-gateway-infra(6aa4d608-50d5-4892-adcd-2432188166f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" podUID="6aa4d608-50d5-4892-adcd-2432188166f4"
Jul 18 01:27:00.926176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:00.926147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:01.031339 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.031304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:01.126805 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.126770    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:01.209745 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.209656    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"]
Jul 18 01:27:01.210066 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.210051    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:01.210111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.210068    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:01.210111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.210079    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:01.210111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.210084    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:01.210203 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.210147    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:01.210203 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.210157    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:01.214499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.214473    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:01.220927 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.220895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"]
Jul 18 01:27:01.352126 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.352089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73fdcfda-42fc-4fc8-be82-9242eceb9d79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t\" (UID: \"73fdcfda-42fc-4fc8-be82-9242eceb9d79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:01.453116 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.453076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73fdcfda-42fc-4fc8-be82-9242eceb9d79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t\" (UID: \"73fdcfda-42fc-4fc8-be82-9242eceb9d79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:01.453937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.453913    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73fdcfda-42fc-4fc8-be82-9242eceb9d79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t\" (UID: \"73fdcfda-42fc-4fc8-be82-9242eceb9d79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:01.528822 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.528738    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:01.683910 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.683877    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"]
Jul 18 01:27:01.686804 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:27:01.686761    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fdcfda_42fc_4fc8_be82_9242eceb9d79.slice/crio-dc3aab59cb203108d2c440625102f33fc450ddc84c5f01a2e158dd6711f73093 WatchSource:0}: Error finding container dc3aab59cb203108d2c440625102f33fc450ddc84c5f01a2e158dd6711f73093: Status 404 returned error can't find the container with id dc3aab59cb203108d2c440625102f33fc450ddc84c5f01a2e158dd6711f73093
Jul 18 01:27:01.988997 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.988957    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" event={"ID":"73fdcfda-42fc-4fc8-be82-9242eceb9d79","Type":"ContainerStarted","Data":"2d76db7e52a5b74bf92c2a261a3ce44fb8d3bca2268d616eda734bebfb9f39a3"}
Jul 18 01:27:01.989162 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:01.989023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" event={"ID":"73fdcfda-42fc-4fc8-be82-9242eceb9d79","Type":"ContainerStarted","Data":"dc3aab59cb203108d2c440625102f33fc450ddc84c5f01a2e158dd6711f73093"}
Jul 18 01:27:02.005013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:02.004957    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" podStartSLOduration=1.004938459 podStartE2EDuration="1.004938459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:27:02.00262236 +0000 UTC m=+2801.530127315" watchObservedRunningTime="2026-07-18 01:27:02.004938459 +0000 UTC m=+2801.532443419"
Jul 18 01:27:02.709774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:02.709681    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:27:02.995459 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:02.995367    2567 generic.go:356] "Generic (PLEG): container finished" podID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerID="2d76db7e52a5b74bf92c2a261a3ce44fb8d3bca2268d616eda734bebfb9f39a3" exitCode=6
Jul 18 01:27:02.995622 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:02.995460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" event={"ID":"73fdcfda-42fc-4fc8-be82-9242eceb9d79","Type":"ContainerDied","Data":"2d76db7e52a5b74bf92c2a261a3ce44fb8d3bca2268d616eda734bebfb9f39a3"}
Jul 18 01:27:02.995622 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:02.995603    2567 scope.go:117] "RemoveContainer" containerID="2d76db7e52a5b74bf92c2a261a3ce44fb8d3bca2268d616eda734bebfb9f39a3"
Jul 18 01:27:04.002157 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:04.002109    2567 generic.go:356] "Generic (PLEG): container finished" podID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerID="210588d9862e4fa88525d7ddb4484d3348cd8da3cb8c39641d7bd2d875d8620a" exitCode=6
Jul 18 01:27:04.002584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:04.002176    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" event={"ID":"73fdcfda-42fc-4fc8-be82-9242eceb9d79","Type":"ContainerDied","Data":"210588d9862e4fa88525d7ddb4484d3348cd8da3cb8c39641d7bd2d875d8620a"}
Jul 18 01:27:04.002584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:04.002236    2567 scope.go:117] "RemoveContainer" containerID="2d76db7e52a5b74bf92c2a261a3ce44fb8d3bca2268d616eda734bebfb9f39a3"
Jul 18 01:27:04.002584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:04.002455    2567 scope.go:117] "RemoveContainer" containerID="210588d9862e4fa88525d7ddb4484d3348cd8da3cb8c39641d7bd2d875d8620a"
Jul 18 01:27:04.002725 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:04.002672    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t_odh-ai-gateway-infra(73fdcfda-42fc-4fc8-be82-9242eceb9d79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79"
Jul 18 01:27:05.009525 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:05.009500    2567 scope.go:117] "RemoveContainer" containerID="210588d9862e4fa88525d7ddb4484d3348cd8da3cb8c39641d7bd2d875d8620a"
Jul 18 01:27:05.009948 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:05.009686    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t_odh-ai-gateway-infra(73fdcfda-42fc-4fc8-be82-9242eceb9d79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79"
Jul 18 01:27:08.920346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:08.913795    2567 scope.go:117] "RemoveContainer" containerID="b91cfab9651d9d88f0dcafe702e4f56cbbe2e3328d3d5094bf309104379ede7d"
Jul 18 01:27:10.036922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:10.036886    2567 generic.go:356] "Generic (PLEG): container finished" podID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerID="d549fbf46618c7ad78dd6b9facfbdb7cd15917f1e492cf899835cd514b6205ec" exitCode=6
Jul 18 01:27:10.037376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:10.036959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" event={"ID":"aff11209-7dcf-49e6-bc60-49d8353c88f5","Type":"ContainerDied","Data":"d549fbf46618c7ad78dd6b9facfbdb7cd15917f1e492cf899835cd514b6205ec"}
Jul 18 01:27:10.037376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:10.037021    2567 scope.go:117] "RemoveContainer" containerID="b91cfab9651d9d88f0dcafe702e4f56cbbe2e3328d3d5094bf309104379ede7d"
Jul 18 01:27:10.037376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:10.037312    2567 scope.go:117] "RemoveContainer" containerID="d549fbf46618c7ad78dd6b9facfbdb7cd15917f1e492cf899835cd514b6205ec"
Jul 18 01:27:10.037535 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:10.037504    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-jcb76_odh-ai-gateway-infra(aff11209-7dcf-49e6-bc60-49d8353c88f5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5"
Jul 18 01:27:11.063831 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.063801    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"]
Jul 18 01:27:11.209282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.209240    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:27:11.228795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.228765    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aff11209-7dcf-49e6-bc60-49d8353c88f5-maas-api-service-ca\") pod \"aff11209-7dcf-49e6-bc60-49d8353c88f5\" (UID: \"aff11209-7dcf-49e6-bc60-49d8353c88f5\") "
Jul 18 01:27:11.229185 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.229161    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff11209-7dcf-49e6-bc60-49d8353c88f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aff11209-7dcf-49e6-bc60-49d8353c88f5" (UID: "aff11209-7dcf-49e6-bc60-49d8353c88f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:27:11.329720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.329626    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aff11209-7dcf-49e6-bc60-49d8353c88f5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:27:11.617435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.617355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:11.913493 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.913404    2567 scope.go:117] "RemoveContainer" containerID="6715402eec8b74f6dec3e5328a8a68edbb0e2a1ac07c56e58d8687f435f33ea4"
Jul 18 01:27:11.929506 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:11.929460    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:12.050916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:12.050881    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"
Jul 18 01:27:12.051068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:12.050923    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76" event={"ID":"aff11209-7dcf-49e6-bc60-49d8353c88f5","Type":"ContainerDied","Data":"20902cb8e63cc43aae895f00c276bd8f91c3076b148c3b38ebf4186ab5110102"}
Jul 18 01:27:12.051068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:12.050991    2567 scope.go:117] "RemoveContainer" containerID="d549fbf46618c7ad78dd6b9facfbdb7cd15917f1e492cf899835cd514b6205ec"
Jul 18 01:27:12.087195 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:12.087163    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"]
Jul 18 01:27:12.092610 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:12.092582    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-jcb76"]
Jul 18 01:27:12.920359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:12.920329    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" path="/var/lib/kubelet/pods/aff11209-7dcf-49e6-bc60-49d8353c88f5/volumes"
Jul 18 01:27:13.058446 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:13.058412    2567 generic.go:356] "Generic (PLEG): container finished" podID="6aa4d608-50d5-4892-adcd-2432188166f4" containerID="835d7212ff1f4fbe1782edd8dd3c5fc1044e202b7e1ff2fdb8515e70e73997c0" exitCode=6
Jul 18 01:27:13.058595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:13.058481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" event={"ID":"6aa4d608-50d5-4892-adcd-2432188166f4","Type":"ContainerDied","Data":"835d7212ff1f4fbe1782edd8dd3c5fc1044e202b7e1ff2fdb8515e70e73997c0"}
Jul 18 01:27:13.058595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:13.058536    2567 scope.go:117] "RemoveContainer" containerID="6715402eec8b74f6dec3e5328a8a68edbb0e2a1ac07c56e58d8687f435f33ea4"
Jul 18 01:27:13.058868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:13.058852    2567 scope.go:117] "RemoveContainer" containerID="835d7212ff1f4fbe1782edd8dd3c5fc1044e202b7e1ff2fdb8515e70e73997c0"
Jul 18 01:27:13.059096 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:13.059074    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf_odh-ai-gateway-infra(6aa4d608-50d5-4892-adcd-2432188166f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" podUID="6aa4d608-50d5-4892-adcd-2432188166f4"
Jul 18 01:27:14.084157 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:14.084120    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"]
Jul 18 01:27:14.227874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:14.227841    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:27:14.257669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:14.257640    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa4d608-50d5-4892-adcd-2432188166f4-maas-api-service-ca\") pod \"6aa4d608-50d5-4892-adcd-2432188166f4\" (UID: \"6aa4d608-50d5-4892-adcd-2432188166f4\") "
Jul 18 01:27:14.258014 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:14.257991    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aa4d608-50d5-4892-adcd-2432188166f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6aa4d608-50d5-4892-adcd-2432188166f4" (UID: "6aa4d608-50d5-4892-adcd-2432188166f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:27:14.359222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:14.359134    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa4d608-50d5-4892-adcd-2432188166f4-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:27:15.071607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:15.071576    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"
Jul 18 01:27:15.071817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:15.071625    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf" event={"ID":"6aa4d608-50d5-4892-adcd-2432188166f4","Type":"ContainerDied","Data":"da38bceef83f1f7b3b084f15b48196a5592f6cedeff3cc37e2dc7d750548cf3d"}
Jul 18 01:27:15.071817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:15.071689    2567 scope.go:117] "RemoveContainer" containerID="835d7212ff1f4fbe1782edd8dd3c5fc1044e202b7e1ff2fdb8515e70e73997c0"
Jul 18 01:27:15.087615 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:15.087590    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"]
Jul 18 01:27:15.093186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:15.093153    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-qklnf"]
Jul 18 01:27:16.919763 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:16.919725    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" path="/var/lib/kubelet/pods/6aa4d608-50d5-4892-adcd-2432188166f4/volumes"
Jul 18 01:27:18.913223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:18.913198    2567 scope.go:117] "RemoveContainer" containerID="210588d9862e4fa88525d7ddb4484d3348cd8da3cb8c39641d7bd2d875d8620a"
Jul 18 01:27:20.106790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:20.106754    2567 generic.go:356] "Generic (PLEG): container finished" podID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerID="a6ebb924ce7989caf51f2f966e945e648891a4d22713b3b2422a12822d85ddca" exitCode=6
Jul 18 01:27:20.107214 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:20.106828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" event={"ID":"73fdcfda-42fc-4fc8-be82-9242eceb9d79","Type":"ContainerDied","Data":"a6ebb924ce7989caf51f2f966e945e648891a4d22713b3b2422a12822d85ddca"}
Jul 18 01:27:20.107214 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:20.106892    2567 scope.go:117] "RemoveContainer" containerID="210588d9862e4fa88525d7ddb4484d3348cd8da3cb8c39641d7bd2d875d8620a"
Jul 18 01:27:20.107214 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:20.107157    2567 scope.go:117] "RemoveContainer" containerID="a6ebb924ce7989caf51f2f966e945e648891a4d22713b3b2422a12822d85ddca"
Jul 18 01:27:20.107427 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:20.107406    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t_odh-ai-gateway-infra(73fdcfda-42fc-4fc8-be82-9242eceb9d79)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79"
Jul 18 01:27:21.134108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:21.134067    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"]
Jul 18 01:27:21.282283 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:21.282247    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:21.326418 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:21.326385    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73fdcfda-42fc-4fc8-be82-9242eceb9d79-maas-api-service-ca\") pod \"73fdcfda-42fc-4fc8-be82-9242eceb9d79\" (UID: \"73fdcfda-42fc-4fc8-be82-9242eceb9d79\") "
Jul 18 01:27:21.326755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:21.326727    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73fdcfda-42fc-4fc8-be82-9242eceb9d79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73fdcfda-42fc-4fc8-be82-9242eceb9d79" (UID: "73fdcfda-42fc-4fc8-be82-9242eceb9d79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:27:21.427908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:21.427819    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73fdcfda-42fc-4fc8-be82-9242eceb9d79-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:27:22.120134 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:22.120103    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"
Jul 18 01:27:22.120362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:22.120103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t" event={"ID":"73fdcfda-42fc-4fc8-be82-9242eceb9d79","Type":"ContainerDied","Data":"dc3aab59cb203108d2c440625102f33fc450ddc84c5f01a2e158dd6711f73093"}
Jul 18 01:27:22.120362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:22.120250    2567 scope.go:117] "RemoveContainer" containerID="a6ebb924ce7989caf51f2f966e945e648891a4d22713b3b2422a12822d85ddca"
Jul 18 01:27:22.139950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:22.139922    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"]
Jul 18 01:27:22.145304 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:22.145262    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-l6q9t"]
Jul 18 01:27:22.920032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:22.919999    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" path="/var/lib/kubelet/pods/73fdcfda-42fc-4fc8-be82-9242eceb9d79/volumes"
Jul 18 01:27:29.072380 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072336    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"]
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072744    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072756    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072764    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072769    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072775    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072781    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072792    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072798    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.072803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072807    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072812    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072826    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072831    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072838    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072843    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072899    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072907    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="026a80a8-a14e-4b74-821a-fd5ef6cfe545" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072915    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072920    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072925    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072931    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072936    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072942    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:29.073125 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.072948    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:29.076019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.076002    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:29.078411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.078388    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:27:29.078535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.078388    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:27:29.083177 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.083148    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"]
Jul 18 01:27:29.197394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.197358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ed6ae2d-b073-4753-95d8-192d0d159a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh\" (UID: \"4ed6ae2d-b073-4753-95d8-192d0d159a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:29.298551 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.298511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ed6ae2d-b073-4753-95d8-192d0d159a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh\" (UID: \"4ed6ae2d-b073-4753-95d8-192d0d159a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:29.299119 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.299096    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ed6ae2d-b073-4753-95d8-192d0d159a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh\" (UID: \"4ed6ae2d-b073-4753-95d8-192d0d159a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:29.388130 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.388024    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:29.733657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:29.733630    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"]
Jul 18 01:27:29.736026 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:27:29.735995    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed6ae2d_b073_4753_95d8_192d0d159a60.slice/crio-45d8ab109839a4f07c2bb321f89da9d33d7ad4cf0bfc0702f08efed6bf508667 WatchSource:0}: Error finding container 45d8ab109839a4f07c2bb321f89da9d33d7ad4cf0bfc0702f08efed6bf508667: Status 404 returned error can't find the container with id 45d8ab109839a4f07c2bb321f89da9d33d7ad4cf0bfc0702f08efed6bf508667
Jul 18 01:27:30.161610 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:30.161574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerStarted","Data":"d47a8ba9d349b33b912fb1569c2b1f351bff68faaef1a03a408c4876657f37f6"}
Jul 18 01:27:30.162080 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:30.161632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerStarted","Data":"45d8ab109839a4f07c2bb321f89da9d33d7ad4cf0bfc0702f08efed6bf508667"}
Jul 18 01:27:30.176869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:30.176815    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" podStartSLOduration=1.176800048 podStartE2EDuration="1.176800048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:27:30.175636685 +0000 UTC m=+2829.703141644" watchObservedRunningTime="2026-07-18 01:27:30.176800048 +0000 UTC m=+2829.704305007"
Jul 18 01:27:31.169202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:31.169163    2567 generic.go:356] "Generic (PLEG): container finished" podID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerID="d47a8ba9d349b33b912fb1569c2b1f351bff68faaef1a03a408c4876657f37f6" exitCode=6
Jul 18 01:27:31.169666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:31.169252    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerDied","Data":"d47a8ba9d349b33b912fb1569c2b1f351bff68faaef1a03a408c4876657f37f6"}
Jul 18 01:27:31.169666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:31.169429    2567 scope.go:117] "RemoveContainer" containerID="d47a8ba9d349b33b912fb1569c2b1f351bff68faaef1a03a408c4876657f37f6"
Jul 18 01:27:32.176171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:32.176137    2567 generic.go:356] "Generic (PLEG): container finished" podID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerID="de594d349598fe042da01cf961e45799729a31ec66fa05fb5171b54e36b57f5f" exitCode=6
Jul 18 01:27:32.176598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:32.176305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerDied","Data":"de594d349598fe042da01cf961e45799729a31ec66fa05fb5171b54e36b57f5f"}
Jul 18 01:27:32.176598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:32.176371    2567 scope.go:117] "RemoveContainer" containerID="d47a8ba9d349b33b912fb1569c2b1f351bff68faaef1a03a408c4876657f37f6"
Jul 18 01:27:32.176598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:32.176596    2567 scope.go:117] "RemoveContainer" containerID="de594d349598fe042da01cf961e45799729a31ec66fa05fb5171b54e36b57f5f"
Jul 18 01:27:32.176872 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:32.176847    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh_odh-ai-gateway-infra(4ed6ae2d-b073-4753-95d8-192d0d159a60)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60"
Jul 18 01:27:32.856077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:32.856038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:33.183216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:33.183137    2567 scope.go:117] "RemoveContainer" containerID="de594d349598fe042da01cf961e45799729a31ec66fa05fb5171b54e36b57f5f"
Jul 18 01:27:33.183610 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:33.183377    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh_odh-ai-gateway-infra(4ed6ae2d-b073-4753-95d8-192d0d159a60)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60"
Jul 18 01:27:38.628334 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:38.628286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:38.729207 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:38.729179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:38.960350 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:38.960246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:39.232842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.232765    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"]
Jul 18 01:27:39.233182 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233169    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:39.233226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233185    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa4d608-50d5-4892-adcd-2432188166f4" containerName="revoke-keys"
Jul 18 01:27:39.233226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233203    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:39.233226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233208    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fdcfda-42fc-4fc8-be82-9242eceb9d79" containerName="revoke-keys"
Jul 18 01:27:39.233226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233219    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:39.233226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233224    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:39.233404 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.233325    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff11209-7dcf-49e6-bc60-49d8353c88f5" containerName="revoke-keys"
Jul 18 01:27:39.237736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.237719    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.240495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.240476    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1e72d0-openshift-default-dockercfg-ph7sh\""
Jul 18 01:27:39.247128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.247104    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"]
Jul 18 01:27:39.264191 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.264160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:39.390130 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390093    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-envoy\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-socket\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-credential-socket\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-certs\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-podinfo\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-token\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-data\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390406    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istiod-ca-cert\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.390618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.390463    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjnzz\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-kube-api-access-rjnzz\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-socket\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491205    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-credential-socket\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-certs\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491262    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-podinfo\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-token\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-data\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istiod-ca-cert\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjnzz\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-kube-api-access-rjnzz\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-envoy\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.491642 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491597    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-socket\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.492021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491663    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-certs\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.492021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-credential-socket\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.492021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.491874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-data\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.492309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.492262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istiod-ca-cert\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.493959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.493934    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-envoy\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.494084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.494066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-podinfo\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.499284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.499241    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-token\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.499397 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.499384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjnzz\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-kube-api-access-rjnzz\") pod \"e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") " pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.549485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.549448    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:39.676850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.676785    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:39.694000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.693963    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"]
Jul 18 01:27:39.696814 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:27:39.696789    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a52e4c7_fe68_4ad9_8b43_e11936c3e3dd.slice/crio-d20aa2a9b1666fdd9b9f8a3d05a3fe9ec6040ebb8bde33d8ad3858b34845f6ae WatchSource:0}: Error finding container d20aa2a9b1666fdd9b9f8a3d05a3fe9ec6040ebb8bde33d8ad3858b34845f6ae: Status 404 returned error can't find the container with id d20aa2a9b1666fdd9b9f8a3d05a3fe9ec6040ebb8bde33d8ad3858b34845f6ae
Jul 18 01:27:39.699221 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.699190    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:39.699335 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.699257    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:39.699335 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:39.699318    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:40.219709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:40.219670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" event={"ID":"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd","Type":"ContainerStarted","Data":"61fb1b48e1418bff366612b93786910d50344a133ac60244fd4f6e5de3ee4d1d"}
Jul 18 01:27:40.219904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:40.219736    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" event={"ID":"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd","Type":"ContainerStarted","Data":"d20aa2a9b1666fdd9b9f8a3d05a3fe9ec6040ebb8bde33d8ad3858b34845f6ae"}
Jul 18 01:27:40.240096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:40.240033    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podStartSLOduration=1.240015768 podStartE2EDuration="1.240015768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:27:40.238058853 +0000 UTC m=+2839.765563814" watchObservedRunningTime="2026-07-18 01:27:40.240015768 +0000 UTC m=+2839.767520727"
Jul 18 01:27:40.550039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:40.549950    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:40.555364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:40.555330    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:41.225129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:41.225101    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:41.226163 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:41.226146    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:27:42.143508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:42.143476    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:42.290817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:42.290786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:42.409814 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:42.409716    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:42.925148 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:42.925118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:43.029427 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.029388    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:43.688065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.688029    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"]
Jul 18 01:27:43.691787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.691767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:27:43.700668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.700641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"]
Jul 18 01:27:43.835708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.835662    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b146bf6-1546-43ef-aee3-ae25522f55cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn\" (UID: \"2b146bf6-1546-43ef-aee3-ae25522f55cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:27:43.936717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.936685    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b146bf6-1546-43ef-aee3-ae25522f55cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn\" (UID: \"2b146bf6-1546-43ef-aee3-ae25522f55cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:27:43.937369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:43.937349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b146bf6-1546-43ef-aee3-ae25522f55cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn\" (UID: \"2b146bf6-1546-43ef-aee3-ae25522f55cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:27:44.004514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:44.004422    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:27:44.147998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:44.147968    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"]
Jul 18 01:27:44.149930 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:27:44.149904    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b146bf6_1546_43ef_aee3_ae25522f55cd.slice/crio-ad077fdd4b42a1893c8c005c727587a09bb17f71e7c4f479cb700d29336c3f5e WatchSource:0}: Error finding container ad077fdd4b42a1893c8c005c727587a09bb17f71e7c4f479cb700d29336c3f5e: Status 404 returned error can't find the container with id ad077fdd4b42a1893c8c005c727587a09bb17f71e7c4f479cb700d29336c3f5e
Jul 18 01:27:44.242280 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:44.242243    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" event={"ID":"2b146bf6-1546-43ef-aee3-ae25522f55cd","Type":"ContainerStarted","Data":"ad077fdd4b42a1893c8c005c727587a09bb17f71e7c4f479cb700d29336c3f5e"}
Jul 18 01:27:45.249995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:45.249894    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerID="64ae39e92cd0cecb807cdc05c9bfbbfcfaadedc9ae452233902db3dd7b0a3398" exitCode=6
Jul 18 01:27:45.250424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:45.249987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" event={"ID":"2b146bf6-1546-43ef-aee3-ae25522f55cd","Type":"ContainerDied","Data":"64ae39e92cd0cecb807cdc05c9bfbbfcfaadedc9ae452233902db3dd7b0a3398"}
Jul 18 01:27:45.250424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:45.250118    2567 scope.go:117] "RemoveContainer" containerID="64ae39e92cd0cecb807cdc05c9bfbbfcfaadedc9ae452233902db3dd7b0a3398"
Jul 18 01:27:46.258528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.258436    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerID="fb950c79ce7bada39903e768386a06b0c5f46c5fb2102ef92d4e868f9978119e" exitCode=6
Jul 18 01:27:46.258975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.258521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" event={"ID":"2b146bf6-1546-43ef-aee3-ae25522f55cd","Type":"ContainerDied","Data":"fb950c79ce7bada39903e768386a06b0c5f46c5fb2102ef92d4e868f9978119e"}
Jul 18 01:27:46.258975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.258586    2567 scope.go:117] "RemoveContainer" containerID="64ae39e92cd0cecb807cdc05c9bfbbfcfaadedc9ae452233902db3dd7b0a3398"
Jul 18 01:27:46.258975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.258848    2567 scope.go:117] "RemoveContainer" containerID="fb950c79ce7bada39903e768386a06b0c5f46c5fb2102ef92d4e868f9978119e"
Jul 18 01:27:46.259119 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:46.259069    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn_odh-ai-gateway-infra(2b146bf6-1546-43ef-aee3-ae25522f55cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd"
Jul 18 01:27:46.287682 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.287650    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"]
Jul 18 01:27:46.292301 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.292281    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:27:46.298123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.298092    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"]
Jul 18 01:27:46.463332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.463258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bdb518-48ac-4d5f-a195-3946f8e46465-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx\" (UID: \"e2bdb518-48ac-4d5f-a195-3946f8e46465\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:27:46.564688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.564640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bdb518-48ac-4d5f-a195-3946f8e46465-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx\" (UID: \"e2bdb518-48ac-4d5f-a195-3946f8e46465\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:27:46.565343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.565323    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bdb518-48ac-4d5f-a195-3946f8e46465-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx\" (UID: \"e2bdb518-48ac-4d5f-a195-3946f8e46465\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:27:46.605856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.605816    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:27:46.637961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.637920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:46.748420 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.748339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"]
Jul 18 01:27:46.750671 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:27:46.750648    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bdb518_48ac_4d5f_a195_3946f8e46465.slice/crio-94f3dd19907db179b9f49ac0f68d21d41604802ee231f91387f028ed17a35643 WatchSource:0}: Error finding container 94f3dd19907db179b9f49ac0f68d21d41604802ee231f91387f028ed17a35643: Status 404 returned error can't find the container with id 94f3dd19907db179b9f49ac0f68d21d41604802ee231f91387f028ed17a35643
Jul 18 01:27:46.948493 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:46.948412    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:47.222594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.222507    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"]
Jul 18 01:27:47.226190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.226166    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.228757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.228735    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8e3c6c-openshift-default-dockercfg-m9sr7\""
Jul 18 01:27:47.237975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.237944    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"]
Jul 18 01:27:47.251976 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.251939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:47.266226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.266186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerStarted","Data":"52fdf1ed3ce7266c77832ea1fec41f229004335df32bd6b339632be3a54f9143"}
Jul 18 01:27:47.266676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.266243    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerStarted","Data":"94f3dd19907db179b9f49ac0f68d21d41604802ee231f91387f028ed17a35643"}
Jul 18 01:27:47.267972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.267954    2567 scope.go:117] "RemoveContainer" containerID="fb950c79ce7bada39903e768386a06b0c5f46c5fb2102ef92d4e868f9978119e"
Jul 18 01:27:47.268159 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:47.268137    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn_odh-ai-gateway-infra(2b146bf6-1546-43ef-aee3-ae25522f55cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd"
Jul 18 01:27:47.271631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-certs\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-socket\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271777 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-podinfo\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271777 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-token\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271777 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271766    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lm9h\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-kube-api-access-4lm9h\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271790    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istiod-ca-cert\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-envoy\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.271902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-data\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.272011 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.271931    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-credential-socket\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.282841 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.282790    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" podStartSLOduration=1.2827724950000001 podStartE2EDuration="1.282772495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:27:47.280760408 +0000 UTC m=+2846.808265394" watchObservedRunningTime="2026-07-18 01:27:47.282772495 +0000 UTC m=+2846.810277454"
Jul 18 01:27:47.372714 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-socket\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.372906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-podinfo\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.372906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-token\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.372906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372783    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lm9h\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-kube-api-access-4lm9h\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.372906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istiod-ca-cert\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.372906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-envoy\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.372906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-data\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.373316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-credential-socket\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.373316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.372941    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-certs\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.373316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.373157    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-socket\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.373316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.373260    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-data\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.373546 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.373332    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-credential-socket\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.373546 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.373384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-certs\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.374103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.374040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istiod-ca-cert\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.375296 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.375262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-envoy\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.375727 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.375707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-podinfo\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.380819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.380796    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-token\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.380907 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.380893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lm9h\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-kube-api-access-4lm9h\") pod \"e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") " pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.537786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.537692    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:47.559782 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.559745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:47.687825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.687797    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"]
Jul 18 01:27:47.690430 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:27:47.690404    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff0c3bc_7207_4f93_bca1_ca6e614f451b.slice/crio-35b62ca9aaaac832285c71a1da8a5cb70acace09802e76931ba8ebd77a87b57f WatchSource:0}: Error finding container 35b62ca9aaaac832285c71a1da8a5cb70acace09802e76931ba8ebd77a87b57f: Status 404 returned error can't find the container with id 35b62ca9aaaac832285c71a1da8a5cb70acace09802e76931ba8ebd77a87b57f
Jul 18 01:27:47.692815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.692783    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:47.692888 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.692855    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:47.692933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.692886    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:47.913527 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:47.913498    2567 scope.go:117] "RemoveContainer" containerID="de594d349598fe042da01cf961e45799729a31ec66fa05fb5171b54e36b57f5f"
Jul 18 01:27:48.274962 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.274880    2567 generic.go:356] "Generic (PLEG): container finished" podID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerID="52fdf1ed3ce7266c77832ea1fec41f229004335df32bd6b339632be3a54f9143" exitCode=6
Jul 18 01:27:48.275429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.274971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerDied","Data":"52fdf1ed3ce7266c77832ea1fec41f229004335df32bd6b339632be3a54f9143"}
Jul 18 01:27:48.275429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.275179    2567 scope.go:117] "RemoveContainer" containerID="52fdf1ed3ce7266c77832ea1fec41f229004335df32bd6b339632be3a54f9143"
Jul 18 01:27:48.277064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.277036    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerStarted","Data":"97200e8e0bd48d36377ff11cc1c50a34f6e75ccc014cda6083bb70078a4e75c2"}
Jul 18 01:27:48.278824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.278807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" event={"ID":"eff0c3bc-7207-4f93-bca1-ca6e614f451b","Type":"ContainerStarted","Data":"cca0eadb75f6ae7fcb3be879e1ce37ba6d65201c6bcf1b4171695a6b1a5e4458"}
Jul 18 01:27:48.278878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.278849    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" event={"ID":"eff0c3bc-7207-4f93-bca1-ca6e614f451b","Type":"ContainerStarted","Data":"35b62ca9aaaac832285c71a1da8a5cb70acace09802e76931ba8ebd77a87b57f"}
Jul 18 01:27:48.309575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.309521    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podStartSLOduration=1.309504117 podStartE2EDuration="1.309504117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:27:48.308981556 +0000 UTC m=+2847.836486511" watchObservedRunningTime="2026-07-18 01:27:48.309504117 +0000 UTC m=+2847.837009077"
Jul 18 01:27:48.538728 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.538633    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:48.543845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:48.543819    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:49.286649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.286562    2567 generic.go:356] "Generic (PLEG): container finished" podID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerID="8a20dc61381da3547baaaa28c65092605f1197250537111781d6ac39311877bf" exitCode=6
Jul 18 01:27:49.287073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.286650    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerDied","Data":"8a20dc61381da3547baaaa28c65092605f1197250537111781d6ac39311877bf"}
Jul 18 01:27:49.287073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.286714    2567 scope.go:117] "RemoveContainer" containerID="52fdf1ed3ce7266c77832ea1fec41f229004335df32bd6b339632be3a54f9143"
Jul 18 01:27:49.287073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.286924    2567 scope.go:117] "RemoveContainer" containerID="8a20dc61381da3547baaaa28c65092605f1197250537111781d6ac39311877bf"
Jul 18 01:27:49.287253 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:49.287148    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx_odh-ai-gateway-infra(e2bdb518-48ac-4d5f-a195-3946f8e46465)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465"
Jul 18 01:27:49.288739 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.288716    2567 generic.go:356] "Generic (PLEG): container finished" podID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerID="97200e8e0bd48d36377ff11cc1c50a34f6e75ccc014cda6083bb70078a4e75c2" exitCode=6
Jul 18 01:27:49.288828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.288746    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerDied","Data":"97200e8e0bd48d36377ff11cc1c50a34f6e75ccc014cda6083bb70078a4e75c2"}
Jul 18 01:27:49.288973 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.288959    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:49.289168 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.289155    2567 scope.go:117] "RemoveContainer" containerID="97200e8e0bd48d36377ff11cc1c50a34f6e75ccc014cda6083bb70078a4e75c2"
Jul 18 01:27:49.289400 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:49.289379    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh_odh-ai-gateway-infra(4ed6ae2d-b073-4753-95d8-192d0d159a60)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60"
Jul 18 01:27:49.290119 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.290099    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:27:49.298629 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.298611    2567 scope.go:117] "RemoveContainer" containerID="de594d349598fe042da01cf961e45799729a31ec66fa05fb5171b54e36b57f5f"
Jul 18 01:27:49.338833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:49.338797    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"]
Jul 18 01:27:50.232882 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.232850    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:50.299165 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.299132    2567 scope.go:117] "RemoveContainer" containerID="8a20dc61381da3547baaaa28c65092605f1197250537111781d6ac39311877bf"
Jul 18 01:27:50.299603 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:27:50.299357    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx_odh-ai-gateway-infra(e2bdb518-48ac-4d5f-a195-3946f8e46465)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465"
Jul 18 01:27:50.317938 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.317893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:50.394486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.394444    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:50.451991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.451961    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:50.596138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.596102    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ed6ae2d-b073-4753-95d8-192d0d159a60-maas-api-service-ca\") pod \"4ed6ae2d-b073-4753-95d8-192d0d159a60\" (UID: \"4ed6ae2d-b073-4753-95d8-192d0d159a60\") "
Jul 18 01:27:50.596531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.596510    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ed6ae2d-b073-4753-95d8-192d0d159a60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ed6ae2d-b073-4753-95d8-192d0d159a60" (UID: "4ed6ae2d-b073-4753-95d8-192d0d159a60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:27:50.627873 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.627828    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:27:50.697146 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.697112    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ed6ae2d-b073-4753-95d8-192d0d159a60-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:27:50.878403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.878316    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:50.990436 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:50.990390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:51.093672 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:51.093622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:51.307189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:51.307163    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"
Jul 18 01:27:51.307684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:51.307191    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh" event={"ID":"4ed6ae2d-b073-4753-95d8-192d0d159a60","Type":"ContainerDied","Data":"45d8ab109839a4f07c2bb321f89da9d33d7ad4cf0bfc0702f08efed6bf508667"}
Jul 18 01:27:51.307684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:51.307251    2567 scope.go:117] "RemoveContainer" containerID="97200e8e0bd48d36377ff11cc1c50a34f6e75ccc014cda6083bb70078a4e75c2"
Jul 18 01:27:51.325510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:51.325475    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"]
Jul 18 01:27:51.329242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:51.329221    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-fsgzh"]
Jul 18 01:27:52.919421 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:52.919385    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" path="/var/lib/kubelet/pods/4ed6ae2d-b073-4753-95d8-192d0d159a60/volumes"
Jul 18 01:27:54.705528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:54.705489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:27:57.265517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:57.265478    2567 patch_prober.go:28] interesting pod/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 18 01:27:57.266026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:57.265559    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 01:27:59.355023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:59.354989    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerID="61fb1b48e1418bff366612b93786910d50344a133ac60244fd4f6e5de3ee4d1d" exitCode=0
Jul 18 01:27:59.355554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:59.355070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" event={"ID":"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd","Type":"ContainerDied","Data":"61fb1b48e1418bff366612b93786910d50344a133ac60244fd4f6e5de3ee4d1d"}
Jul 18 01:27:59.355554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:59.355524    2567 scope.go:117] "RemoveContainer" containerID="61fb1b48e1418bff366612b93786910d50344a133ac60244fd4f6e5de3ee4d1d"
Jul 18 01:27:59.356560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:59.356527    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:59.356665 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:59.356600    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:27:59.356665 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:27:59.356640    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:28:00.366256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:00.366207    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" event={"ID":"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd","Type":"ContainerStarted","Data":"d72ee04aad99712c526ad16d8ec2782be17bdef68a09ff71bf0e8cc7841ce293"}
Jul 18 01:28:00.550748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:00.550710    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:28:00.551235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:00.551142    2567 patch_prober.go:28] interesting pod/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Jul 18 01:28:00.551493 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:00.551467    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Jul 18 01:28:00.913049 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:00.913025    2567 scope.go:117] "RemoveContainer" containerID="fb950c79ce7bada39903e768386a06b0c5f46c5fb2102ef92d4e868f9978119e"
Jul 18 01:28:01.261376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.261333    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"]
Jul 18 01:28:01.262381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.262344    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.262562 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.262551    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.262671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.262661    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.262766 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.262757    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.262887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.262878    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.262981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.262973    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.263222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.263211    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.263362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.263351    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:01.267681 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.267651    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:01.301037 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.300986    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh\" (UID: \"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:01.316083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.316049    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"]
Jul 18 01:28:01.391308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.387919    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" event={"ID":"2b146bf6-1546-43ef-aee3-ae25522f55cd","Type":"ContainerStarted","Data":"41bf7ecf1d19710f4d964b694e04ec5022c7b3c75a33622a7197dea34e382fc2"}
Jul 18 01:28:01.405299 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.403931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh\" (UID: \"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:01.405299 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.405131    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh\" (UID: \"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:01.411174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.411128    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" podStartSLOduration=18.411112777 podStartE2EDuration="18.411112777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:28:01.408420099 +0000 UTC m=+2860.935925060" watchObservedRunningTime="2026-07-18 01:28:01.411112777 +0000 UTC m=+2860.938617728"
Jul 18 01:28:01.550057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.549961    2567 patch_prober.go:28] interesting pod/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Jul 18 01:28:01.550057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.550021    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Jul 18 01:28:01.586952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.586889    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:01.832900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:01.832872    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"]
Jul 18 01:28:02.400874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.400762    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerStarted","Data":"b9db861c4cb63b6ef6643f742f8e9d09f7599920f2f44c036aa0f25fb254cf3d"}
Jul 18 01:28:02.400874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.400840    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerStarted","Data":"b562ecc31eb8493d1af9f99bf43c99fb6f60d6019d180fd2d9d751f2a2711659"}
Jul 18 01:28:02.404200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.404161    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerID="41bf7ecf1d19710f4d964b694e04ec5022c7b3c75a33622a7197dea34e382fc2" exitCode=6
Jul 18 01:28:02.404465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.404303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" event={"ID":"2b146bf6-1546-43ef-aee3-ae25522f55cd","Type":"ContainerDied","Data":"41bf7ecf1d19710f4d964b694e04ec5022c7b3c75a33622a7197dea34e382fc2"}
Jul 18 01:28:02.404465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.404362    2567 scope.go:117] "RemoveContainer" containerID="fb950c79ce7bada39903e768386a06b0c5f46c5fb2102ef92d4e868f9978119e"
Jul 18 01:28:02.404787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.404765    2567 scope.go:117] "RemoveContainer" containerID="41bf7ecf1d19710f4d964b694e04ec5022c7b3c75a33622a7197dea34e382fc2"
Jul 18 01:28:02.405014 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:02.404987    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn_odh-ai-gateway-infra(2b146bf6-1546-43ef-aee3-ae25522f55cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd"
Jul 18 01:28:02.418547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.418495    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" podStartSLOduration=1.4184761639999999 podStartE2EDuration="1.418476164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:28:02.416100488 +0000 UTC m=+2861.943605448" watchObservedRunningTime="2026-07-18 01:28:02.418476164 +0000 UTC m=+2861.945981139"
Jul 18 01:28:02.421566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.421545    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"]
Jul 18 01:28:02.550789 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.550737    2567 patch_prober.go:28] interesting pod/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Jul 18 01:28:02.551007 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:02.550813    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Jul 18 01:28:03.413805 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.413770    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerID="b9db861c4cb63b6ef6643f742f8e9d09f7599920f2f44c036aa0f25fb254cf3d" exitCode=6
Jul 18 01:28:03.414097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.413823    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerDied","Data":"b9db861c4cb63b6ef6643f742f8e9d09f7599920f2f44c036aa0f25fb254cf3d"}
Jul 18 01:28:03.414159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.414142    2567 scope.go:117] "RemoveContainer" containerID="b9db861c4cb63b6ef6643f742f8e9d09f7599920f2f44c036aa0f25fb254cf3d"
Jul 18 01:28:03.554756 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.554712    2567 patch_prober.go:28] interesting pod/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Jul 18 01:28:03.555055 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.555022    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Jul 18 01:28:03.667343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.663420    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:03.689595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.689573    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:28:03.734572 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.734532    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b146bf6-1546-43ef-aee3-ae25522f55cd-maas-api-service-ca\") pod \"2b146bf6-1546-43ef-aee3-ae25522f55cd\" (UID: \"2b146bf6-1546-43ef-aee3-ae25522f55cd\") "
Jul 18 01:28:03.734897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.734875    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b146bf6-1546-43ef-aee3-ae25522f55cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b146bf6-1546-43ef-aee3-ae25522f55cd" (UID: "2b146bf6-1546-43ef-aee3-ae25522f55cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:28:03.776295 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.775755    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:03.836028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.835986    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b146bf6-1546-43ef-aee3-ae25522f55cd-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:28:03.913252 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:03.913218    2567 scope.go:117] "RemoveContainer" containerID="8a20dc61381da3547baaaa28c65092605f1197250537111781d6ac39311877bf"
Jul 18 01:28:04.422588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.422552    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerID="672356e498678b5aea5f6b24bb51d1e3f7cce62dd808d208e90c9d9beb7adcdc" exitCode=6
Jul 18 01:28:04.423056 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.422649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerDied","Data":"672356e498678b5aea5f6b24bb51d1e3f7cce62dd808d208e90c9d9beb7adcdc"}
Jul 18 01:28:04.423056 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.422717    2567 scope.go:117] "RemoveContainer" containerID="b9db861c4cb63b6ef6643f742f8e9d09f7599920f2f44c036aa0f25fb254cf3d"
Jul 18 01:28:04.423056 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.422836    2567 scope.go:117] "RemoveContainer" containerID="672356e498678b5aea5f6b24bb51d1e3f7cce62dd808d208e90c9d9beb7adcdc"
Jul 18 01:28:04.423190 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:04.423057    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh_odh-ai-gateway-infra(8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5"
Jul 18 01:28:04.424668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.424650    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"
Jul 18 01:28:04.424806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.424700    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn" event={"ID":"2b146bf6-1546-43ef-aee3-ae25522f55cd","Type":"ContainerDied","Data":"ad077fdd4b42a1893c8c005c727587a09bb17f71e7c4f479cb700d29336c3f5e"}
Jul 18 01:28:04.427017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.426956    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerStarted","Data":"b373ae2ffb7b35edfda87132e5786d33e8b9356e012951041b790b68afc05273"}
Jul 18 01:28:04.438060 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.437972    2567 scope.go:117] "RemoveContainer" containerID="41bf7ecf1d19710f4d964b694e04ec5022c7b3c75a33622a7197dea34e382fc2"
Jul 18 01:28:04.449848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.449824    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"]
Jul 18 01:28:04.454614 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.454578    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-b9vrn"]
Jul 18 01:28:04.555684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.555587    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:28:04.556154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.556126    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:28:04.556914 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.556895    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:28:04.911938 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.911903    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:28:04.920309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:04.920255    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" path="/var/lib/kubelet/pods/2b146bf6-1546-43ef-aee3-ae25522f55cd/volumes"
Jul 18 01:28:05.330518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.330471    2567 patch_prober.go:28] interesting pod/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 18 01:28:05.330745 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.330541    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 01:28:05.433876 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.433846    2567 scope.go:117] "RemoveContainer" containerID="672356e498678b5aea5f6b24bb51d1e3f7cce62dd808d208e90c9d9beb7adcdc"
Jul 18 01:28:05.434341 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:05.434106    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh_odh-ai-gateway-infra(8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5"
Jul 18 01:28:05.436763 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.436733    2567 generic.go:356] "Generic (PLEG): container finished" podID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerID="b373ae2ffb7b35edfda87132e5786d33e8b9356e012951041b790b68afc05273" exitCode=6
Jul 18 01:28:05.436912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.436812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerDied","Data":"b373ae2ffb7b35edfda87132e5786d33e8b9356e012951041b790b68afc05273"}
Jul 18 01:28:05.436912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.436875    2567 scope.go:117] "RemoveContainer" containerID="8a20dc61381da3547baaaa28c65092605f1197250537111781d6ac39311877bf"
Jul 18 01:28:05.437322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.437303    2567 scope.go:117] "RemoveContainer" containerID="b373ae2ffb7b35edfda87132e5786d33e8b9356e012951041b790b68afc05273"
Jul 18 01:28:05.437542 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:05.437519    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx_odh-ai-gateway-infra(e2bdb518-48ac-4d5f-a195-3946f8e46465)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465"
Jul 18 01:28:05.472776 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:05.472734    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"]
Jul 18 01:28:06.596417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:06.596389    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:28:06.665337 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:06.665304    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bdb518-48ac-4d5f-a195-3946f8e46465-maas-api-service-ca\") pod \"e2bdb518-48ac-4d5f-a195-3946f8e46465\" (UID: \"e2bdb518-48ac-4d5f-a195-3946f8e46465\") "
Jul 18 01:28:06.665739 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:06.665716    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2bdb518-48ac-4d5f-a195-3946f8e46465-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2bdb518-48ac-4d5f-a195-3946f8e46465" (UID: "e2bdb518-48ac-4d5f-a195-3946f8e46465"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:28:06.766702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:06.766624    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2bdb518-48ac-4d5f-a195-3946f8e46465-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:28:07.449538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.449506    2567 generic.go:356] "Generic (PLEG): container finished" podID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerID="cca0eadb75f6ae7fcb3be879e1ce37ba6d65201c6bcf1b4171695a6b1a5e4458" exitCode=0
Jul 18 01:28:07.449813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.449580    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" event={"ID":"eff0c3bc-7207-4f93-bca1-ca6e614f451b","Type":"ContainerDied","Data":"cca0eadb75f6ae7fcb3be879e1ce37ba6d65201c6bcf1b4171695a6b1a5e4458"}
Jul 18 01:28:07.450060 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.450043    2567 scope.go:117] "RemoveContainer" containerID="cca0eadb75f6ae7fcb3be879e1ce37ba6d65201c6bcf1b4171695a6b1a5e4458"
Jul 18 01:28:07.451201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.451178    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx" event={"ID":"e2bdb518-48ac-4d5f-a195-3946f8e46465","Type":"ContainerDied","Data":"94f3dd19907db179b9f49ac0f68d21d41604802ee231f91387f028ed17a35643"}
Jul 18 01:28:07.451332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.451237    2567 scope.go:117] "RemoveContainer" containerID="b373ae2ffb7b35edfda87132e5786d33e8b9356e012951041b790b68afc05273"
Jul 18 01:28:07.451332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.451190    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"
Jul 18 01:28:07.451429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.451174    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:28:07.451563 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.451495    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:28:07.451637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.451558    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:28:07.485772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.485729    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"]
Jul 18 01:28:07.487683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:07.487659    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-t6jwx"]
Jul 18 01:28:08.464134 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:08.464083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" event={"ID":"eff0c3bc-7207-4f93-bca1-ca6e614f451b","Type":"ContainerStarted","Data":"332eb553438c9ca2a18f7d4df415ac8eae6d7b488c42894bc133d45031bccd8a"}
Jul 18 01:28:08.537950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:08.537911    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:28:08.538193 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:08.538172    2567 patch_prober.go:28] interesting pod/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Jul 18 01:28:08.538242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:08.538219    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Jul 18 01:28:08.919917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:08.919878    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" path="/var/lib/kubelet/pods/e2bdb518-48ac-4d5f-a195-3946f8e46465/volumes"
Jul 18 01:28:09.538664 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:09.538626    2567 patch_prober.go:28] interesting pod/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Jul 18 01:28:09.539073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:09.538694    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Jul 18 01:28:10.538860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:10.538819    2567 patch_prober.go:28] interesting pod/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Jul 18 01:28:10.539263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:10.538892    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Jul 18 01:28:11.538472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:11.538389    2567 patch_prober.go:28] interesting pod/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Jul 18 01:28:11.538472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:11.538447    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Jul 18 01:28:11.736350 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:11.736313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:11.840558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:11.840526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:12.541485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:12.541454    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:28:12.541754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:12.541735    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:28:12.542396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:12.542377    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:28:15.119896 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:15.119861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:17.913223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:17.913196    2567 scope.go:117] "RemoveContainer" containerID="672356e498678b5aea5f6b24bb51d1e3f7cce62dd808d208e90c9d9beb7adcdc"
Jul 18 01:28:18.523510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:18.523417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerStarted","Data":"4a5e071b86539c51fa3e0494dcd3564e6bdffbe205586b2492b3eec3ab361817"}
Jul 18 01:28:18.759405 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:18.759359    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:28:19.530630 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:19.530595    2567 generic.go:356] "Generic (PLEG): container finished" podID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerID="4a5e071b86539c51fa3e0494dcd3564e6bdffbe205586b2492b3eec3ab361817" exitCode=6
Jul 18 01:28:19.531071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:19.530666    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerDied","Data":"4a5e071b86539c51fa3e0494dcd3564e6bdffbe205586b2492b3eec3ab361817"}
Jul 18 01:28:19.531071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:19.530728    2567 scope.go:117] "RemoveContainer" containerID="672356e498678b5aea5f6b24bb51d1e3f7cce62dd808d208e90c9d9beb7adcdc"
Jul 18 01:28:19.531071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:19.531066    2567 scope.go:117] "RemoveContainer" containerID="4a5e071b86539c51fa3e0494dcd3564e6bdffbe205586b2492b3eec3ab361817"
Jul 18 01:28:19.531324 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:19.531302    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh_odh-ai-gateway-infra(8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5"
Jul 18 01:28:19.551977 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:19.551951    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"]
Jul 18 01:28:20.684748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:20.684720    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:20.810495 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:20.810458    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5-maas-api-service-ca\") pod \"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5\" (UID: \"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5\") "
Jul 18 01:28:20.810821 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:20.810796    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" (UID: "8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:28:20.911172 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:20.911135    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:28:21.543466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:21.543436    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"
Jul 18 01:28:21.543631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:21.543472    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh" event={"ID":"8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5","Type":"ContainerDied","Data":"b562ecc31eb8493d1af9f99bf43c99fb6f60d6019d180fd2d9d751f2a2711659"}
Jul 18 01:28:21.543631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:21.543536    2567 scope.go:117] "RemoveContainer" containerID="4a5e071b86539c51fa3e0494dcd3564e6bdffbe205586b2492b3eec3ab361817"
Jul 18 01:28:21.559497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:21.559465    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"]
Jul 18 01:28:21.562258 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:21.562227    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-strlh"]
Jul 18 01:28:22.919636 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:22.919600    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" path="/var/lib/kubelet/pods/8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5/volumes"
Jul 18 01:28:29.113201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"]
Jul 18 01:28:29.113596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113584    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:29.113596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113595    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113608    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113613    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113623    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113630    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113642    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113648    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113657    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113662    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113669    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.113687 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113674    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113735    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113742    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113748    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113754    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ed6ae2d-b073-4753-95d8-192d0d159a60" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113760    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113774    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113782    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:29.114013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.113787    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:29.116800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.116783    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:29.119155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.119127    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:28:29.119309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.119148    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:28:29.126634 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.126102    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"]
Jul 18 01:28:29.288709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.288664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3358330f-c547-4f3b-b881-dccd87510fa8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc\" (UID: \"3358330f-c547-4f3b-b881-dccd87510fa8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:29.390210 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.390111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3358330f-c547-4f3b-b881-dccd87510fa8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc\" (UID: \"3358330f-c547-4f3b-b881-dccd87510fa8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:29.390787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.390766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3358330f-c547-4f3b-b881-dccd87510fa8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc\" (UID: \"3358330f-c547-4f3b-b881-dccd87510fa8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:29.429176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.429138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:29.570864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.570836    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"]
Jul 18 01:28:29.572638 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:28:29.572600    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3358330f_c547_4f3b_b881_dccd87510fa8.slice/crio-87882578a612f77b4a9217de958b4955ea6578e69a230d92e05636ee838d1f61 WatchSource:0}: Error finding container 87882578a612f77b4a9217de958b4955ea6578e69a230d92e05636ee838d1f61: Status 404 returned error can't find the container with id 87882578a612f77b4a9217de958b4955ea6578e69a230d92e05636ee838d1f61
Jul 18 01:28:29.590233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:29.590199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" event={"ID":"3358330f-c547-4f3b-b881-dccd87510fa8","Type":"ContainerStarted","Data":"87882578a612f77b4a9217de958b4955ea6578e69a230d92e05636ee838d1f61"}
Jul 18 01:28:30.596737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:30.596692    2567 generic.go:356] "Generic (PLEG): container finished" podID="3358330f-c547-4f3b-b881-dccd87510fa8" containerID="155e3cf41b593f3a2d9ccf1c94b5feb5bd94e420cf31551db9210780e0af59d7" exitCode=6
Jul 18 01:28:30.597233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:30.596783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" event={"ID":"3358330f-c547-4f3b-b881-dccd87510fa8","Type":"ContainerDied","Data":"155e3cf41b593f3a2d9ccf1c94b5feb5bd94e420cf31551db9210780e0af59d7"}
Jul 18 01:28:30.597233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:30.596983    2567 scope.go:117] "RemoveContainer" containerID="155e3cf41b593f3a2d9ccf1c94b5feb5bd94e420cf31551db9210780e0af59d7"
Jul 18 01:28:31.604206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:31.604175    2567 generic.go:356] "Generic (PLEG): container finished" podID="3358330f-c547-4f3b-b881-dccd87510fa8" containerID="a12fdec5952a002960fbd3924fa0c8913e30a594aa34764f8bd65ff9bda39774" exitCode=6
Jul 18 01:28:31.604674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:31.604260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" event={"ID":"3358330f-c547-4f3b-b881-dccd87510fa8","Type":"ContainerDied","Data":"a12fdec5952a002960fbd3924fa0c8913e30a594aa34764f8bd65ff9bda39774"}
Jul 18 01:28:31.604674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:31.604342    2567 scope.go:117] "RemoveContainer" containerID="155e3cf41b593f3a2d9ccf1c94b5feb5bd94e420cf31551db9210780e0af59d7"
Jul 18 01:28:31.604674 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:31.604536    2567 scope.go:117] "RemoveContainer" containerID="a12fdec5952a002960fbd3924fa0c8913e30a594aa34764f8bd65ff9bda39774"
Jul 18 01:28:31.604818 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:31.604773    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc_odh-ai-gateway-infra(3358330f-c547-4f3b-b881-dccd87510fa8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" podUID="3358330f-c547-4f3b-b881-dccd87510fa8"
Jul 18 01:28:32.611976 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:32.611951    2567 scope.go:117] "RemoveContainer" containerID="a12fdec5952a002960fbd3924fa0c8913e30a594aa34764f8bd65ff9bda39774"
Jul 18 01:28:32.612428 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:32.612240    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc_odh-ai-gateway-infra(3358330f-c547-4f3b-b881-dccd87510fa8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" podUID="3358330f-c547-4f3b-b881-dccd87510fa8"
Jul 18 01:28:37.949374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:37.949340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:42.407988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:42.407948    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:28:43.731972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.731935    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"]
Jul 18 01:28:43.732377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732342    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:43.732377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732355    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:43.732377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732377    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:43.732497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732382    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:43.732497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732390    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:43.732497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732397    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cbbfd9f-c87e-4b06-a02e-b4c04e5b88b5" containerName="revoke-keys"
Jul 18 01:28:43.732497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732463    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b146bf6-1546-43ef-aee3-ae25522f55cd" containerName="revoke-keys"
Jul 18 01:28:43.732497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.732476    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2bdb518-48ac-4d5f-a195-3946f8e46465" containerName="revoke-keys"
Jul 18 01:28:43.736859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.736841    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:28:43.742521 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.742479    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"]
Jul 18 01:28:43.812366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.812329    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3980bec-b50b-4fc6-b783-b77360ade454-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm\" (UID: \"b3980bec-b50b-4fc6-b783-b77360ade454\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:28:43.913810 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.913754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3980bec-b50b-4fc6-b783-b77360ade454-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm\" (UID: \"b3980bec-b50b-4fc6-b783-b77360ade454\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:28:43.914424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:43.914404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3980bec-b50b-4fc6-b783-b77360ade454-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm\" (UID: \"b3980bec-b50b-4fc6-b783-b77360ade454\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:28:44.050435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:44.050333    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:28:44.195670 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:44.195639    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"]
Jul 18 01:28:44.197547 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:28:44.197515    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3980bec_b50b_4fc6_b783_b77360ade454.slice/crio-324c6dc6bb2ea83bd1fc754b646233e2af9f222962be72c1de0c16083533f487 WatchSource:0}: Error finding container 324c6dc6bb2ea83bd1fc754b646233e2af9f222962be72c1de0c16083533f487: Status 404 returned error can't find the container with id 324c6dc6bb2ea83bd1fc754b646233e2af9f222962be72c1de0c16083533f487
Jul 18 01:28:44.676170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:44.676066    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" event={"ID":"b3980bec-b50b-4fc6-b783-b77360ade454","Type":"ContainerStarted","Data":"d235d4b447d3dfdccd63d6c9e1098fea30415b9b6f16ae20aaa14c7ffe8bf59b"}
Jul 18 01:28:44.676170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:44.676126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" event={"ID":"b3980bec-b50b-4fc6-b783-b77360ade454","Type":"ContainerStarted","Data":"324c6dc6bb2ea83bd1fc754b646233e2af9f222962be72c1de0c16083533f487"}
Jul 18 01:28:44.690779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:44.690729    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" podStartSLOduration=1.690713022 podStartE2EDuration="1.690713022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:28:44.689881044 +0000 UTC m=+2904.217386005" watchObservedRunningTime="2026-07-18 01:28:44.690713022 +0000 UTC m=+2904.218217982"
Jul 18 01:28:44.914067 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:44.914039    2567 scope.go:117] "RemoveContainer" containerID="a12fdec5952a002960fbd3924fa0c8913e30a594aa34764f8bd65ff9bda39774"
Jul 18 01:28:45.682827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.682779    2567 generic.go:356] "Generic (PLEG): container finished" podID="3358330f-c547-4f3b-b881-dccd87510fa8" containerID="3679d09acdc2fa24467789c822f44a664c22cad9edde26547db043c98367b444" exitCode=6
Jul 18 01:28:45.683018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.682853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" event={"ID":"3358330f-c547-4f3b-b881-dccd87510fa8","Type":"ContainerDied","Data":"3679d09acdc2fa24467789c822f44a664c22cad9edde26547db043c98367b444"}
Jul 18 01:28:45.683018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.682918    2567 scope.go:117] "RemoveContainer" containerID="a12fdec5952a002960fbd3924fa0c8913e30a594aa34764f8bd65ff9bda39774"
Jul 18 01:28:45.683322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.683302    2567 scope.go:117] "RemoveContainer" containerID="3679d09acdc2fa24467789c822f44a664c22cad9edde26547db043c98367b444"
Jul 18 01:28:45.683600 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:45.683575    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc_odh-ai-gateway-infra(3358330f-c547-4f3b-b881-dccd87510fa8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" podUID="3358330f-c547-4f3b-b881-dccd87510fa8"
Jul 18 01:28:45.685009 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.684986    2567 generic.go:356] "Generic (PLEG): container finished" podID="b3980bec-b50b-4fc6-b783-b77360ade454" containerID="d235d4b447d3dfdccd63d6c9e1098fea30415b9b6f16ae20aaa14c7ffe8bf59b" exitCode=6
Jul 18 01:28:45.685097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.685056    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" event={"ID":"b3980bec-b50b-4fc6-b783-b77360ade454","Type":"ContainerDied","Data":"d235d4b447d3dfdccd63d6c9e1098fea30415b9b6f16ae20aaa14c7ffe8bf59b"}
Jul 18 01:28:45.685257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:45.685245    2567 scope.go:117] "RemoveContainer" containerID="d235d4b447d3dfdccd63d6c9e1098fea30415b9b6f16ae20aaa14c7ffe8bf59b"
Jul 18 01:28:46.330001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.329957    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"]
Jul 18 01:28:46.333773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.333738    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:28:46.341078 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.341039    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"]
Jul 18 01:28:46.439552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.439521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90abedab-e359-4f47-86f5-48b34c7ba8ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2\" (UID: \"90abedab-e359-4f47-86f5-48b34c7ba8ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:28:46.540297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.540247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90abedab-e359-4f47-86f5-48b34c7ba8ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2\" (UID: \"90abedab-e359-4f47-86f5-48b34c7ba8ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:28:46.540869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.540847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90abedab-e359-4f47-86f5-48b34c7ba8ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2\" (UID: \"90abedab-e359-4f47-86f5-48b34c7ba8ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:28:46.646688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.646595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:28:46.695602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.695569    2567 generic.go:356] "Generic (PLEG): container finished" podID="b3980bec-b50b-4fc6-b783-b77360ade454" containerID="dcd9a476c8c493176e2b86d435010dbd1036585e55af382f827cd5aafc1482d3" exitCode=6
Jul 18 01:28:46.695781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.695653    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" event={"ID":"b3980bec-b50b-4fc6-b783-b77360ade454","Type":"ContainerDied","Data":"dcd9a476c8c493176e2b86d435010dbd1036585e55af382f827cd5aafc1482d3"}
Jul 18 01:28:46.695781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.695707    2567 scope.go:117] "RemoveContainer" containerID="d235d4b447d3dfdccd63d6c9e1098fea30415b9b6f16ae20aaa14c7ffe8bf59b"
Jul 18 01:28:46.695917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.695899    2567 scope.go:117] "RemoveContainer" containerID="dcd9a476c8c493176e2b86d435010dbd1036585e55af382f827cd5aafc1482d3"
Jul 18 01:28:46.696223 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:46.696192    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm_odh-ai-gateway-infra(b3980bec-b50b-4fc6-b783-b77360ade454)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" podUID="b3980bec-b50b-4fc6-b783-b77360ade454"
Jul 18 01:28:46.712326 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.712202    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"]
Jul 18 01:28:46.800994 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.800962    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"]
Jul 18 01:28:46.867121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.867099    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:46.943499 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.943412    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3358330f-c547-4f3b-b881-dccd87510fa8-maas-api-service-ca\") pod \"3358330f-c547-4f3b-b881-dccd87510fa8\" (UID: \"3358330f-c547-4f3b-b881-dccd87510fa8\") "
Jul 18 01:28:46.943781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:46.943760    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3358330f-c547-4f3b-b881-dccd87510fa8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3358330f-c547-4f3b-b881-dccd87510fa8" (UID: "3358330f-c547-4f3b-b881-dccd87510fa8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:28:47.044883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.044841    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3358330f-c547-4f3b-b881-dccd87510fa8-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:28:47.702748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.702714    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"
Jul 18 01:28:47.703187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.702748    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc" event={"ID":"3358330f-c547-4f3b-b881-dccd87510fa8","Type":"ContainerDied","Data":"87882578a612f77b4a9217de958b4955ea6578e69a230d92e05636ee838d1f61"}
Jul 18 01:28:47.703187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.702809    2567 scope.go:117] "RemoveContainer" containerID="3679d09acdc2fa24467789c822f44a664c22cad9edde26547db043c98367b444"
Jul 18 01:28:47.704922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.704900    2567 scope.go:117] "RemoveContainer" containerID="dcd9a476c8c493176e2b86d435010dbd1036585e55af382f827cd5aafc1482d3"
Jul 18 01:28:47.705208 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:47.705176    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm_odh-ai-gateway-infra(b3980bec-b50b-4fc6-b783-b77360ade454)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" podUID="b3980bec-b50b-4fc6-b783-b77360ade454"
Jul 18 01:28:47.706377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.706357    2567 generic.go:356] "Generic (PLEG): container finished" podID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerID="088ab7bd34d73405026c3b8dd9815f8dcd174423974a91dd1f8477a15fba904b" exitCode=6
Jul 18 01:28:47.706498 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.706400    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" event={"ID":"90abedab-e359-4f47-86f5-48b34c7ba8ba","Type":"ContainerDied","Data":"088ab7bd34d73405026c3b8dd9815f8dcd174423974a91dd1f8477a15fba904b"}
Jul 18 01:28:47.706498 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.706467    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" event={"ID":"90abedab-e359-4f47-86f5-48b34c7ba8ba","Type":"ContainerStarted","Data":"1106d9a3da10986ea971d461133de511c2a3171f46d62bcb8599506906060097"}
Jul 18 01:28:47.706630 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.706614    2567 scope.go:117] "RemoveContainer" containerID="088ab7bd34d73405026c3b8dd9815f8dcd174423974a91dd1f8477a15fba904b"
Jul 18 01:28:47.748947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.748579    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"]
Jul 18 01:28:47.756597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:47.756398    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-xc5vc"]
Jul 18 01:28:48.715654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:48.715564    2567 generic.go:356] "Generic (PLEG): container finished" podID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerID="558390db3469fdd587121310db3d236ff5fd0d5cf58c0043cb0d4ef7b740b90d" exitCode=6
Jul 18 01:28:48.716058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:48.715658    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" event={"ID":"90abedab-e359-4f47-86f5-48b34c7ba8ba","Type":"ContainerDied","Data":"558390db3469fdd587121310db3d236ff5fd0d5cf58c0043cb0d4ef7b740b90d"}
Jul 18 01:28:48.716058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:48.715724    2567 scope.go:117] "RemoveContainer" containerID="088ab7bd34d73405026c3b8dd9815f8dcd174423974a91dd1f8477a15fba904b"
Jul 18 01:28:48.716058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:48.715955    2567 scope.go:117] "RemoveContainer" containerID="558390db3469fdd587121310db3d236ff5fd0d5cf58c0043cb0d4ef7b740b90d"
Jul 18 01:28:48.716250 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:48.716224    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2_odh-ai-gateway-infra(90abedab-e359-4f47-86f5-48b34c7ba8ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba"
Jul 18 01:28:48.919283 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:48.919231    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" path="/var/lib/kubelet/pods/3358330f-c547-4f3b-b881-dccd87510fa8/volumes"
Jul 18 01:28:49.724091 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:49.724062    2567 scope.go:117] "RemoveContainer" containerID="558390db3469fdd587121310db3d236ff5fd0d5cf58c0043cb0d4ef7b740b90d"
Jul 18 01:28:49.724542 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:49.724259    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2_odh-ai-gateway-infra(90abedab-e359-4f47-86f5-48b34c7ba8ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba"
Jul 18 01:28:56.217742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:56.217698    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:28:58.131746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:58.131716    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:58.914014 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:58.913981    2567 scope.go:117] "RemoveContainer" containerID="dcd9a476c8c493176e2b86d435010dbd1036585e55af382f827cd5aafc1482d3"
Jul 18 01:28:58.986314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:58.986247    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:59.113447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113412    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"]
Jul 18 01:28:59.113872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113853    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.113872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113871    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.113969 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113891    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.113969 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113897    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.114030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113970    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.114030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113979    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.114030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.113988    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:28:59.118241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.118224    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:28:59.123138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.123114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"]
Jul 18 01:28:59.266671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.266632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdf5d8f5-802c-4773-8327-65085b7f8ea6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn\" (UID: \"bdf5d8f5-802c-4773-8327-65085b7f8ea6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:28:59.367808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.367773    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdf5d8f5-802c-4773-8327-65085b7f8ea6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn\" (UID: \"bdf5d8f5-802c-4773-8327-65085b7f8ea6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:28:59.368439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.368419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdf5d8f5-802c-4773-8327-65085b7f8ea6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn\" (UID: \"bdf5d8f5-802c-4773-8327-65085b7f8ea6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:28:59.429542 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.429507    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:28:59.577466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.577376    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"]
Jul 18 01:28:59.580020 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:28:59.579989    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf5d8f5_802c_4773_8327_65085b7f8ea6.slice/crio-a5df97239984f8089b0e10ffbdd51aacac8b63e0b241b703b47fbc9b9c6aff4e WatchSource:0}: Error finding container a5df97239984f8089b0e10ffbdd51aacac8b63e0b241b703b47fbc9b9c6aff4e: Status 404 returned error can't find the container with id a5df97239984f8089b0e10ffbdd51aacac8b63e0b241b703b47fbc9b9c6aff4e
Jul 18 01:28:59.608189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.608072    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:59.708788 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.708749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:28:59.781992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.781898    2567 generic.go:356] "Generic (PLEG): container finished" podID="b3980bec-b50b-4fc6-b783-b77360ade454" containerID="4060480d602f09a4115008524f68770265a7b0815bd5a534a8037fd3b8e24ee3" exitCode=6
Jul 18 01:28:59.781992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.781981    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" event={"ID":"b3980bec-b50b-4fc6-b783-b77360ade454","Type":"ContainerDied","Data":"4060480d602f09a4115008524f68770265a7b0815bd5a534a8037fd3b8e24ee3"}
Jul 18 01:28:59.782225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.782047    2567 scope.go:117] "RemoveContainer" containerID="dcd9a476c8c493176e2b86d435010dbd1036585e55af382f827cd5aafc1482d3"
Jul 18 01:28:59.782384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.782368    2567 scope.go:117] "RemoveContainer" containerID="4060480d602f09a4115008524f68770265a7b0815bd5a534a8037fd3b8e24ee3"
Jul 18 01:28:59.782732 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:28:59.782702    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm_odh-ai-gateway-infra(b3980bec-b50b-4fc6-b783-b77360ade454)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" podUID="b3980bec-b50b-4fc6-b783-b77360ade454"
Jul 18 01:28:59.783753 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:28:59.783733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn" event={"ID":"bdf5d8f5-802c-4773-8327-65085b7f8ea6","Type":"ContainerStarted","Data":"a5df97239984f8089b0e10ffbdd51aacac8b63e0b241b703b47fbc9b9c6aff4e"}
Jul 18 01:29:00.791104 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:00.791015    2567 generic.go:356] "Generic (PLEG): container finished" podID="bdf5d8f5-802c-4773-8327-65085b7f8ea6" containerID="8924caf7be3cd7fac3c47c9dfce959650e3f1e767dbcc3db211f47f566ebe2f8" exitCode=0
Jul 18 01:29:00.791563 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:00.791115    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn" event={"ID":"bdf5d8f5-802c-4773-8327-65085b7f8ea6","Type":"ContainerDied","Data":"8924caf7be3cd7fac3c47c9dfce959650e3f1e767dbcc3db211f47f566ebe2f8"}
Jul 18 01:29:00.806731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:00.806701    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"]
Jul 18 01:29:00.950684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:00.950653    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:29:01.084098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.084059    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3980bec-b50b-4fc6-b783-b77360ade454-maas-api-service-ca\") pod \"b3980bec-b50b-4fc6-b783-b77360ade454\" (UID: \"b3980bec-b50b-4fc6-b783-b77360ade454\") "
Jul 18 01:29:01.084507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.084475    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3980bec-b50b-4fc6-b783-b77360ade454-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3980bec-b50b-4fc6-b783-b77360ade454" (UID: "b3980bec-b50b-4fc6-b783-b77360ade454"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:01.185676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.185643    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3980bec-b50b-4fc6-b783-b77360ade454-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:01.305048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305006    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"]
Jul 18 01:29:01.305456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305442    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:01.305517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305459    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:01.305517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305473    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:01.305517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305479    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:01.305517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305498    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:29:01.305517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305505    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3358330f-c547-4f3b-b881-dccd87510fa8" containerName="revoke-keys"
Jul 18 01:29:01.305689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305563    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:01.305689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.305574    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:01.308731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.308715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:01.318144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.318114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"]
Jul 18 01:29:01.387742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.387647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a10c91c-a98e-4c64-959e-61f0520b859a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb\" (UID: \"6a10c91c-a98e-4c64-959e-61f0520b859a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:01.489021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.488978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a10c91c-a98e-4c64-959e-61f0520b859a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb\" (UID: \"6a10c91c-a98e-4c64-959e-61f0520b859a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:01.489922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.489892    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a10c91c-a98e-4c64-959e-61f0520b859a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb\" (UID: \"6a10c91c-a98e-4c64-959e-61f0520b859a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:01.619866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.619825    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:01.773547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.773439    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"]
Jul 18 01:29:01.776565 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:29:01.776531    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a10c91c_a98e_4c64_959e_61f0520b859a.slice/crio-3212caea97189503b25c05d85bbbeda47c09c87e0714f725e65e9be0d3ee3692 WatchSource:0}: Error finding container 3212caea97189503b25c05d85bbbeda47c09c87e0714f725e65e9be0d3ee3692: Status 404 returned error can't find the container with id 3212caea97189503b25c05d85bbbeda47c09c87e0714f725e65e9be0d3ee3692
Jul 18 01:29:01.800785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.800753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" event={"ID":"6a10c91c-a98e-4c64-959e-61f0520b859a","Type":"ContainerStarted","Data":"3212caea97189503b25c05d85bbbeda47c09c87e0714f725e65e9be0d3ee3692"}
Jul 18 01:29:01.802127 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.802088    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm" event={"ID":"b3980bec-b50b-4fc6-b783-b77360ade454","Type":"ContainerDied","Data":"324c6dc6bb2ea83bd1fc754b646233e2af9f222962be72c1de0c16083533f487"}
Jul 18 01:29:01.802236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.802161    2567 scope.go:117] "RemoveContainer" containerID="4060480d602f09a4115008524f68770265a7b0815bd5a534a8037fd3b8e24ee3"
Jul 18 01:29:01.802236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.802116    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"
Jul 18 01:29:01.832366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.832333    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"]
Jul 18 01:29:01.836957 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.836930    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-ncfxm"]
Jul 18 01:29:01.913425 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.913394    2567 scope.go:117] "RemoveContainer" containerID="558390db3469fdd587121310db3d236ff5fd0d5cf58c0043cb0d4ef7b740b90d"
Jul 18 01:29:01.946686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:01.946661    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:29:02.094500 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.094466    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdf5d8f5-802c-4773-8327-65085b7f8ea6-maas-api-service-ca\") pod \"bdf5d8f5-802c-4773-8327-65085b7f8ea6\" (UID: \"bdf5d8f5-802c-4773-8327-65085b7f8ea6\") "
Jul 18 01:29:02.094830 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.094808    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdf5d8f5-802c-4773-8327-65085b7f8ea6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bdf5d8f5-802c-4773-8327-65085b7f8ea6" (UID: "bdf5d8f5-802c-4773-8327-65085b7f8ea6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:02.196058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.195970    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdf5d8f5-802c-4773-8327-65085b7f8ea6-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:02.809624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.809590    2567 generic.go:356] "Generic (PLEG): container finished" podID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerID="2edfdf2c0c749e2eea880d79ddc34b9cb80aff007f9f57dddcf64e499787738d" exitCode=6
Jul 18 01:29:02.810115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.809661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" event={"ID":"90abedab-e359-4f47-86f5-48b34c7ba8ba","Type":"ContainerDied","Data":"2edfdf2c0c749e2eea880d79ddc34b9cb80aff007f9f57dddcf64e499787738d"}
Jul 18 01:29:02.810115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.809736    2567 scope.go:117] "RemoveContainer" containerID="558390db3469fdd587121310db3d236ff5fd0d5cf58c0043cb0d4ef7b740b90d"
Jul 18 01:29:02.810115 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.810039    2567 scope.go:117] "RemoveContainer" containerID="2edfdf2c0c749e2eea880d79ddc34b9cb80aff007f9f57dddcf64e499787738d"
Jul 18 01:29:02.810343 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:02.810322    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2_odh-ai-gateway-infra(90abedab-e359-4f47-86f5-48b34c7ba8ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba"
Jul 18 01:29:02.811322 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.811302    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerID="8d199105786db1fdf6d3ffb6dfbd025f143ea737fed67059d036b3f70cd16a1c" exitCode=6
Jul 18 01:29:02.811455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.811436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" event={"ID":"6a10c91c-a98e-4c64-959e-61f0520b859a","Type":"ContainerDied","Data":"8d199105786db1fdf6d3ffb6dfbd025f143ea737fed67059d036b3f70cd16a1c"}
Jul 18 01:29:02.811598 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.811583    2567 scope.go:117] "RemoveContainer" containerID="8d199105786db1fdf6d3ffb6dfbd025f143ea737fed67059d036b3f70cd16a1c"
Jul 18 01:29:02.813851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.813832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn" event={"ID":"bdf5d8f5-802c-4773-8327-65085b7f8ea6","Type":"ContainerDied","Data":"a5df97239984f8089b0e10ffbdd51aacac8b63e0b241b703b47fbc9b9c6aff4e"}
Jul 18 01:29:02.813933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.813867    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5df97239984f8089b0e10ffbdd51aacac8b63e0b241b703b47fbc9b9c6aff4e"
Jul 18 01:29:02.813933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.813906    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn"
Jul 18 01:29:02.920230 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:02.920188    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" path="/var/lib/kubelet/pods/b3980bec-b50b-4fc6-b783-b77360ade454/volumes"
Jul 18 01:29:03.822892 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:03.822859    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerID="a39d0b295e3bf8a4a07e9ecb129daba6857753d906d4b607d40912b8c21fa5c3" exitCode=6
Jul 18 01:29:03.823384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:03.822945    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" event={"ID":"6a10c91c-a98e-4c64-959e-61f0520b859a","Type":"ContainerDied","Data":"a39d0b295e3bf8a4a07e9ecb129daba6857753d906d4b607d40912b8c21fa5c3"}
Jul 18 01:29:03.823384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:03.823003    2567 scope.go:117] "RemoveContainer" containerID="8d199105786db1fdf6d3ffb6dfbd025f143ea737fed67059d036b3f70cd16a1c"
Jul 18 01:29:03.823384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:03.823146    2567 scope.go:117] "RemoveContainer" containerID="a39d0b295e3bf8a4a07e9ecb129daba6857753d906d4b607d40912b8c21fa5c3"
Jul 18 01:29:03.823530 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:03.823390    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb_odh-ai-gateway-infra(6a10c91c-a98e-4c64-959e-61f0520b859a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a"
Jul 18 01:29:03.834699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:03.834664    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"]
Jul 18 01:29:04.005043 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.005019    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:29:04.113582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.113495    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90abedab-e359-4f47-86f5-48b34c7ba8ba-maas-api-service-ca\") pod \"90abedab-e359-4f47-86f5-48b34c7ba8ba\" (UID: \"90abedab-e359-4f47-86f5-48b34c7ba8ba\") "
Jul 18 01:29:04.113881 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.113856    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90abedab-e359-4f47-86f5-48b34c7ba8ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90abedab-e359-4f47-86f5-48b34c7ba8ba" (UID: "90abedab-e359-4f47-86f5-48b34c7ba8ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:04.215159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.215118    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90abedab-e359-4f47-86f5-48b34c7ba8ba-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:04.829476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.829449    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"
Jul 18 01:29:04.829967 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.829448    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2" event={"ID":"90abedab-e359-4f47-86f5-48b34c7ba8ba","Type":"ContainerDied","Data":"1106d9a3da10986ea971d461133de511c2a3171f46d62bcb8599506906060097"}
Jul 18 01:29:04.829967 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.829606    2567 scope.go:117] "RemoveContainer" containerID="2edfdf2c0c749e2eea880d79ddc34b9cb80aff007f9f57dddcf64e499787738d"
Jul 18 01:29:04.831481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.831460    2567 scope.go:117] "RemoveContainer" containerID="a39d0b295e3bf8a4a07e9ecb129daba6857753d906d4b607d40912b8c21fa5c3"
Jul 18 01:29:04.831699 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:04.831678    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb_odh-ai-gateway-infra(6a10c91c-a98e-4c64-959e-61f0520b859a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a"
Jul 18 01:29:04.861445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.861404    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"]
Jul 18 01:29:04.863905 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.863877    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-58sm2"]
Jul 18 01:29:04.919194 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:04.919152    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" path="/var/lib/kubelet/pods/90abedab-e359-4f47-86f5-48b34c7ba8ba/volumes"
Jul 18 01:29:09.553571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:09.553534    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:09.803004 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:09.802962    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:29:14.199477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:14.199446    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:18.913663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:18.913631    2567 scope.go:117] "RemoveContainer" containerID="a39d0b295e3bf8a4a07e9ecb129daba6857753d906d4b607d40912b8c21fa5c3"
Jul 18 01:29:19.158160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158124    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"]
Jul 18 01:29:19.158734 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158718    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158737    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158745    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158751    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158761    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdf5d8f5-802c-4773-8327-65085b7f8ea6" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158767    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdf5d8f5-802c-4773-8327-65085b7f8ea6" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158774    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158779    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158786    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158792    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158871    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158880    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158886    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90abedab-e359-4f47-86f5-48b34c7ba8ba" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158896    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3980bec-b50b-4fc6-b783-b77360ade454" containerName="revoke-keys"
Jul 18 01:29:19.158998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.158903    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdf5d8f5-802c-4773-8327-65085b7f8ea6" containerName="revoke-keys"
Jul 18 01:29:19.163219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.163199    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:19.170535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.170316    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"]
Jul 18 01:29:19.250156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.250118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8\" (UID: \"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:19.350963 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.350927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8\" (UID: \"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:19.351992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.351970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8\" (UID: \"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:19.483841 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.483759    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:19.623956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.623930    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"]
Jul 18 01:29:19.625824 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:29:19.625801    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdd3d8a_dfac_46c2_a7f6_eb927bc8d5bf.slice/crio-bc5609a86378aad64609c5dd9202bfad1c5a343357ce885ec26702379cacd8b4 WatchSource:0}: Error finding container bc5609a86378aad64609c5dd9202bfad1c5a343357ce885ec26702379cacd8b4: Status 404 returned error can't find the container with id bc5609a86378aad64609c5dd9202bfad1c5a343357ce885ec26702379cacd8b4
Jul 18 01:29:19.913344 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.913304    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" event={"ID":"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf","Type":"ContainerStarted","Data":"bc5609a86378aad64609c5dd9202bfad1c5a343357ce885ec26702379cacd8b4"}
Jul 18 01:29:19.915327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.915299    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerID="15ab6ef4ca5b0a5596a6eeab636ba96c4288068a7fb9e93b21bb3b6a826f4340" exitCode=6
Jul 18 01:29:19.915754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.915335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" event={"ID":"6a10c91c-a98e-4c64-959e-61f0520b859a","Type":"ContainerDied","Data":"15ab6ef4ca5b0a5596a6eeab636ba96c4288068a7fb9e93b21bb3b6a826f4340"}
Jul 18 01:29:19.915754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.915385    2567 scope.go:117] "RemoveContainer" containerID="a39d0b295e3bf8a4a07e9ecb129daba6857753d906d4b607d40912b8c21fa5c3"
Jul 18 01:29:19.915754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.915622    2567 scope.go:117] "RemoveContainer" containerID="15ab6ef4ca5b0a5596a6eeab636ba96c4288068a7fb9e93b21bb3b6a826f4340"
Jul 18 01:29:19.915882 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:19.915848    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb_odh-ai-gateway-infra(6a10c91c-a98e-4c64-959e-61f0520b859a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a"
Jul 18 01:29:19.943646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:19.943609    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:29:20.231627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:20.231540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:20.331529 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:20.331495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:20.922205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:20.922169    2567 generic.go:356] "Generic (PLEG): container finished" podID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerID="7509eef35a2ab1ff6b99f56ebcd6d2021fa6f8c2e32435b188ef6a331e0f2779" exitCode=6
Jul 18 01:29:20.922732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:20.922208    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" event={"ID":"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf","Type":"ContainerDied","Data":"7509eef35a2ab1ff6b99f56ebcd6d2021fa6f8c2e32435b188ef6a331e0f2779"}
Jul 18 01:29:20.922732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:20.922464    2567 scope.go:117] "RemoveContainer" containerID="7509eef35a2ab1ff6b99f56ebcd6d2021fa6f8c2e32435b188ef6a331e0f2779"
Jul 18 01:29:20.942905 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:20.942862    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"]
Jul 18 01:29:21.096900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.096874    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:21.170373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.170342    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a10c91c-a98e-4c64-959e-61f0520b859a-maas-api-service-ca\") pod \"6a10c91c-a98e-4c64-959e-61f0520b859a\" (UID: \"6a10c91c-a98e-4c64-959e-61f0520b859a\") "
Jul 18 01:29:21.170732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.170708    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a10c91c-a98e-4c64-959e-61f0520b859a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a10c91c-a98e-4c64-959e-61f0520b859a" (UID: "6a10c91c-a98e-4c64-959e-61f0520b859a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:21.271757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.271728    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a10c91c-a98e-4c64-959e-61f0520b859a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:21.931599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.931566    2567 generic.go:356] "Generic (PLEG): container finished" podID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerID="eb0b1010d7897c92fe90aa410b3f903d7b5993060bfe0478b2861c42017b2639" exitCode=6
Jul 18 01:29:21.932063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.931650    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" event={"ID":"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf","Type":"ContainerDied","Data":"eb0b1010d7897c92fe90aa410b3f903d7b5993060bfe0478b2861c42017b2639"}
Jul 18 01:29:21.932063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.931724    2567 scope.go:117] "RemoveContainer" containerID="7509eef35a2ab1ff6b99f56ebcd6d2021fa6f8c2e32435b188ef6a331e0f2779"
Jul 18 01:29:21.932063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.931951    2567 scope.go:117] "RemoveContainer" containerID="eb0b1010d7897c92fe90aa410b3f903d7b5993060bfe0478b2861c42017b2639"
Jul 18 01:29:21.932202 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:21.932182    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8_odh-ai-gateway-infra(9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf"
Jul 18 01:29:21.933077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.933058    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb" event={"ID":"6a10c91c-a98e-4c64-959e-61f0520b859a","Type":"ContainerDied","Data":"3212caea97189503b25c05d85bbbeda47c09c87e0714f725e65e9be0d3ee3692"}
Jul 18 01:29:21.933174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.933107    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"
Jul 18 01:29:21.943604 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.943583    2567 scope.go:117] "RemoveContainer" containerID="15ab6ef4ca5b0a5596a6eeab636ba96c4288068a7fb9e93b21bb3b6a826f4340"
Jul 18 01:29:21.959690 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.959661    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"]
Jul 18 01:29:21.963860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:21.963831    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gdfkb"]
Jul 18 01:29:22.919236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:22.919206    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" path="/var/lib/kubelet/pods/6a10c91c-a98e-4c64-959e-61f0520b859a/volumes"
Jul 18 01:29:22.941180 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:22.941157    2567 scope.go:117] "RemoveContainer" containerID="eb0b1010d7897c92fe90aa410b3f903d7b5993060bfe0478b2861c42017b2639"
Jul 18 01:29:22.941571 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:22.941395    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8_odh-ai-gateway-infra(9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf"
Jul 18 01:29:24.834342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:24.834298    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"]
Jul 18 01:29:24.834889 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:24.834585    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy" containerID="cri-o://d72ee04aad99712c526ad16d8ec2782be17bdef68a09ff71bf0e8cc7841ce293" gracePeriod=30
Jul 18 01:29:24.853310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:24.853223    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:25.074560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.074528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:25.188176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188089    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"]
Jul 18 01:29:25.188496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188481    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.188496 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188496    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.188592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188517    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.188592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188523    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.188592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188584    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.188592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188591    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.188729 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.188598    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:25.193475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.193452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:25.199859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.199825    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"]
Jul 18 01:29:25.307633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.307593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/273feba5-634a-4b53-8d3e-bf57e12f2800-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng\" (UID: \"273feba5-634a-4b53-8d3e-bf57e12f2800\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:25.409185 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.409128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/273feba5-634a-4b53-8d3e-bf57e12f2800-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng\" (UID: \"273feba5-634a-4b53-8d3e-bf57e12f2800\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:25.409809 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.409787    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/273feba5-634a-4b53-8d3e-bf57e12f2800-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng\" (UID: \"273feba5-634a-4b53-8d3e-bf57e12f2800\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:25.505913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.505825    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:25.649139 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.649113    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"]
Jul 18 01:29:25.651557 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:29:25.651530    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273feba5_634a_4b53_8d3e_bf57e12f2800.slice/crio-61795e53641825518b028b45d6c97fc39a637350b718f235abcf3c9c88b25fe8 WatchSource:0}: Error finding container 61795e53641825518b028b45d6c97fc39a637350b718f235abcf3c9c88b25fe8: Status 404 returned error can't find the container with id 61795e53641825518b028b45d6c97fc39a637350b718f235abcf3c9c88b25fe8
Jul 18 01:29:25.684930 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.684888    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:25.788347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.788243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:25.958321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:25.958285    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng" event={"ID":"273feba5-634a-4b53-8d3e-bf57e12f2800","Type":"ContainerStarted","Data":"61795e53641825518b028b45d6c97fc39a637350b718f235abcf3c9c88b25fe8"}
Jul 18 01:29:26.965171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:26.965140    2567 generic.go:356] "Generic (PLEG): container finished" podID="273feba5-634a-4b53-8d3e-bf57e12f2800" containerID="3dc453351e3a2ac32f05af2a4322ac3b04687fa03436ee1492ee7602f9f765ec" exitCode=0
Jul 18 01:29:26.965645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:26.965232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng" event={"ID":"273feba5-634a-4b53-8d3e-bf57e12f2800","Type":"ContainerDied","Data":"3dc453351e3a2ac32f05af2a4322ac3b04687fa03436ee1492ee7602f9f765ec"}
Jul 18 01:29:28.118263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.118234    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:28.233107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.233026    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/273feba5-634a-4b53-8d3e-bf57e12f2800-maas-api-service-ca\") pod \"273feba5-634a-4b53-8d3e-bf57e12f2800\" (UID: \"273feba5-634a-4b53-8d3e-bf57e12f2800\") "
Jul 18 01:29:28.233456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.233433    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/273feba5-634a-4b53-8d3e-bf57e12f2800-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "273feba5-634a-4b53-8d3e-bf57e12f2800" (UID: "273feba5-634a-4b53-8d3e-bf57e12f2800"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:28.334114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.334072    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/273feba5-634a-4b53-8d3e-bf57e12f2800-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:28.978803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.978772    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng"
Jul 18 01:29:28.978954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.978807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng" event={"ID":"273feba5-634a-4b53-8d3e-bf57e12f2800","Type":"ContainerDied","Data":"61795e53641825518b028b45d6c97fc39a637350b718f235abcf3c9c88b25fe8"}
Jul 18 01:29:28.978954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:28.978860    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61795e53641825518b028b45d6c97fc39a637350b718f235abcf3c9c88b25fe8"
Jul 18 01:29:29.986485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:29.986457    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerID="d72ee04aad99712c526ad16d8ec2782be17bdef68a09ff71bf0e8cc7841ce293" exitCode=0
Jul 18 01:29:29.986935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:29.986516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" event={"ID":"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd","Type":"ContainerDied","Data":"d72ee04aad99712c526ad16d8ec2782be17bdef68a09ff71bf0e8cc7841ce293"}
Jul 18 01:29:29.986935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:29.986566    2567 scope.go:117] "RemoveContainer" containerID="61fb1b48e1418bff366612b93786910d50344a133ac60244fd4f6e5de3ee4d1d"
Jul 18 01:29:30.111910 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.111877    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:29:30.150477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150441    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-podinfo\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150479    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjnzz\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-kube-api-access-rjnzz\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150507    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-socket\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150538    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-data\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150656    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-envoy\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-credential-socket\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150713    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istiod-ca-cert\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.150742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-token\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.151048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150768    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-certs\") pod \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\" (UID: \"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd\") "
Jul 18 01:29:30.151048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.150946    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:30.151048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.151033    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:30.151213 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.151110    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.151213 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.151195    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:30.151774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.151709    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:30.152485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.152449    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:30.153191 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.153162    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-kube-api-access-rjnzz" (OuterVolumeSpecName: "kube-api-access-rjnzz") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "kube-api-access-rjnzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:29:30.153365 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.153334    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:29:30.153624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.153601    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:30.153692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.153643    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" (UID: "6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:29:30.252041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252003    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252035    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252045    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252054    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252062    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252071    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252079    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjnzz\" (UniqueName: \"kubernetes.io/projected/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-kube-api-access-rjnzz\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.252318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.252087    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:30.993535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.993502    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"
Jul 18 01:29:30.993988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.993536    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl" event={"ID":"6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd","Type":"ContainerDied","Data":"d20aa2a9b1666fdd9b9f8a3d05a3fe9ec6040ebb8bde33d8ad3858b34845f6ae"}
Jul 18 01:29:30.993988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:30.993596    2567 scope.go:117] "RemoveContainer" containerID="d72ee04aad99712c526ad16d8ec2782be17bdef68a09ff71bf0e8cc7841ce293"
Jul 18 01:29:31.011071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:31.011043    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"]
Jul 18 01:29:31.014899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:31.014874    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1e72d0-openshift-default-74969d7c55-c6drl"]
Jul 18 01:29:32.919329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:32.919295    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" path="/var/lib/kubelet/pods/6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd/volumes"
Jul 18 01:29:33.582543 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.582512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:33.683246 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.683213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:33.771699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.771665    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"]
Jul 18 01:29:33.772047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772034    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:33.772094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772049    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a10c91c-a98e-4c64-959e-61f0520b859a" containerName="revoke-keys"
Jul 18 01:29:33.772094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772065    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy"
Jul 18 01:29:33.772094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772071    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy"
Jul 18 01:29:33.772094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772081    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273feba5-634a-4b53-8d3e-bf57e12f2800" containerName="revoke-keys"
Jul 18 01:29:33.772094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772086    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="273feba5-634a-4b53-8d3e-bf57e12f2800" containerName="revoke-keys"
Jul 18 01:29:33.772247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772100    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy"
Jul 18 01:29:33.772247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772105    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy"
Jul 18 01:29:33.772247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772162    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="273feba5-634a-4b53-8d3e-bf57e12f2800" containerName="revoke-keys"
Jul 18 01:29:33.772247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772173    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy"
Jul 18 01:29:33.772247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.772179    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a52e4c7-fe68-4ad9-8b43-e11936c3e3dd" containerName="istio-proxy"
Jul 18 01:29:33.776573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.776553    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:33.782068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.782038    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"]
Jul 18 01:29:33.883012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.882920    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf\" (UID: \"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:33.983855 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.983819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf\" (UID: \"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:33.984505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:33.984485    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf\" (UID: \"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:34.089801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:34.089736    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:34.231834 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:34.231804    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"]
Jul 18 01:29:34.233686 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:29:34.233648    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff565e0_ee9c_4ee4_99ec_23c0f7cdae9b.slice/crio-3c2d52330611880f0bde55aa2e15aa0462540122f30152c9f5f78f4eb2b3953b WatchSource:0}: Error finding container 3c2d52330611880f0bde55aa2e15aa0462540122f30152c9f5f78f4eb2b3953b: Status 404 returned error can't find the container with id 3c2d52330611880f0bde55aa2e15aa0462540122f30152c9f5f78f4eb2b3953b
Jul 18 01:29:35.016859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:35.016822    2567 generic.go:356] "Generic (PLEG): container finished" podID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerID="53ac01e1c42b1c48ebbf2248e35e9a50c5f0470849a892895ea4921dc043bbca" exitCode=6
Jul 18 01:29:35.017338 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:35.016915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" event={"ID":"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b","Type":"ContainerDied","Data":"53ac01e1c42b1c48ebbf2248e35e9a50c5f0470849a892895ea4921dc043bbca"}
Jul 18 01:29:35.017338 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:35.016974    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" event={"ID":"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b","Type":"ContainerStarted","Data":"3c2d52330611880f0bde55aa2e15aa0462540122f30152c9f5f78f4eb2b3953b"}
Jul 18 01:29:35.017338 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:35.017100    2567 scope.go:117] "RemoveContainer" containerID="53ac01e1c42b1c48ebbf2248e35e9a50c5f0470849a892895ea4921dc043bbca"
Jul 18 01:29:35.677735 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:35.677699    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:36.024085 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.024048    2567 generic.go:356] "Generic (PLEG): container finished" podID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerID="0f5b8a191b82a2b0eaea19e776d9523d30db8aade2fa5d57ad746cc6d0cc1423" exitCode=6
Jul 18 01:29:36.024516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.024141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" event={"ID":"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b","Type":"ContainerDied","Data":"0f5b8a191b82a2b0eaea19e776d9523d30db8aade2fa5d57ad746cc6d0cc1423"}
Jul 18 01:29:36.024516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.024199    2567 scope.go:117] "RemoveContainer" containerID="53ac01e1c42b1c48ebbf2248e35e9a50c5f0470849a892895ea4921dc043bbca"
Jul 18 01:29:36.024516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.024368    2567 scope.go:117] "RemoveContainer" containerID="0f5b8a191b82a2b0eaea19e776d9523d30db8aade2fa5d57ad746cc6d0cc1423"
Jul 18 01:29:36.024661 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:36.024603    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf_odh-ai-gateway-infra(dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b"
Jul 18 01:29:36.382289 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.382234    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"]
Jul 18 01:29:36.386980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.386956    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:36.394813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.394528    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"]
Jul 18 01:29:36.507600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.507564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cadb17-b743-469d-8da0-89a737101d4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-drw72\" (UID: \"82cadb17-b743-469d-8da0-89a737101d4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:36.608334 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.608254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cadb17-b743-469d-8da0-89a737101d4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-drw72\" (UID: \"82cadb17-b743-469d-8da0-89a737101d4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:36.608907 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.608884    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cadb17-b743-469d-8da0-89a737101d4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-drw72\" (UID: \"82cadb17-b743-469d-8da0-89a737101d4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:36.699379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.699261    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:36.845072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:36.845047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"]
Jul 18 01:29:36.847027 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:29:36.846998    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82cadb17_b743_469d_8da0_89a737101d4c.slice/crio-bad5f6f7b64bad1ef7005b31ff8d4a1921da7a0cb19b773e758c96754150245e WatchSource:0}: Error finding container bad5f6f7b64bad1ef7005b31ff8d4a1921da7a0cb19b773e758c96754150245e: Status 404 returned error can't find the container with id bad5f6f7b64bad1ef7005b31ff8d4a1921da7a0cb19b773e758c96754150245e
Jul 18 01:29:37.031790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:37.031705    2567 scope.go:117] "RemoveContainer" containerID="0f5b8a191b82a2b0eaea19e776d9523d30db8aade2fa5d57ad746cc6d0cc1423"
Jul 18 01:29:37.032235 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:37.031965    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf_odh-ai-gateway-infra(dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b"
Jul 18 01:29:37.032774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:37.032751    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" event={"ID":"82cadb17-b743-469d-8da0-89a737101d4c","Type":"ContainerStarted","Data":"bad5f6f7b64bad1ef7005b31ff8d4a1921da7a0cb19b773e758c96754150245e"}
Jul 18 01:29:37.102998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:37.102958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:37.913989 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:37.913892    2567 scope.go:117] "RemoveContainer" containerID="eb0b1010d7897c92fe90aa410b3f903d7b5993060bfe0478b2861c42017b2639"
Jul 18 01:29:38.040351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:38.040313    2567 generic.go:356] "Generic (PLEG): container finished" podID="82cadb17-b743-469d-8da0-89a737101d4c" containerID="1f53e8a827c6da1bcf43f67524b2483cb5a541a8fc3b2f9fafca42c9c1a1cce6" exitCode=6
Jul 18 01:29:38.040762 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:38.040398    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" event={"ID":"82cadb17-b743-469d-8da0-89a737101d4c","Type":"ContainerDied","Data":"1f53e8a827c6da1bcf43f67524b2483cb5a541a8fc3b2f9fafca42c9c1a1cce6"}
Jul 18 01:29:38.040762 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:38.040605    2567 scope.go:117] "RemoveContainer" containerID="1f53e8a827c6da1bcf43f67524b2483cb5a541a8fc3b2f9fafca42c9c1a1cce6"
Jul 18 01:29:38.710768 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:38.710730    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:29:39.047941 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.047854    2567 generic.go:356] "Generic (PLEG): container finished" podID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerID="bb53509191a5de1879b91d2f7039804198250b268eda1130fc97c19e83038963" exitCode=6
Jul 18 01:29:39.047941 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.047923    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" event={"ID":"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf","Type":"ContainerDied","Data":"bb53509191a5de1879b91d2f7039804198250b268eda1130fc97c19e83038963"}
Jul 18 01:29:39.048479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.047978    2567 scope.go:117] "RemoveContainer" containerID="eb0b1010d7897c92fe90aa410b3f903d7b5993060bfe0478b2861c42017b2639"
Jul 18 01:29:39.048479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.048323    2567 scope.go:117] "RemoveContainer" containerID="bb53509191a5de1879b91d2f7039804198250b268eda1130fc97c19e83038963"
Jul 18 01:29:39.048569 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:39.048554    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8_odh-ai-gateway-infra(9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf"
Jul 18 01:29:39.050121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.050101    2567 generic.go:356] "Generic (PLEG): container finished" podID="82cadb17-b743-469d-8da0-89a737101d4c" containerID="dc6d0cbb52af5614958bb7d27780d07745bf809651fc9a68e81db2d3770cc795" exitCode=6
Jul 18 01:29:39.050232 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.050145    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" event={"ID":"82cadb17-b743-469d-8da0-89a737101d4c","Type":"ContainerDied","Data":"dc6d0cbb52af5614958bb7d27780d07745bf809651fc9a68e81db2d3770cc795"}
Jul 18 01:29:39.050417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.050402    2567 scope.go:117] "RemoveContainer" containerID="dc6d0cbb52af5614958bb7d27780d07745bf809651fc9a68e81db2d3770cc795"
Jul 18 01:29:39.050585 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:39.050567    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-drw72_odh-ai-gateway-infra(82cadb17-b743-469d-8da0-89a737101d4c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" podUID="82cadb17-b743-469d-8da0-89a737101d4c"
Jul 18 01:29:39.060898 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:39.060876    2567 scope.go:117] "RemoveContainer" containerID="1f53e8a827c6da1bcf43f67524b2483cb5a541a8fc3b2f9fafca42c9c1a1cce6"
Jul 18 01:29:40.060557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.060529    2567 scope.go:117] "RemoveContainer" containerID="dc6d0cbb52af5614958bb7d27780d07745bf809651fc9a68e81db2d3770cc795"
Jul 18 01:29:40.061026 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:40.060828    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-drw72_odh-ai-gateway-infra(82cadb17-b743-469d-8da0-89a737101d4c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" podUID="82cadb17-b743-469d-8da0-89a737101d4c"
Jul 18 01:29:40.071953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.071921    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"]
Jul 18 01:29:40.222006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.221981    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:40.277506 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.277475    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:40.344163 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.344072    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf-maas-api-service-ca\") pod \"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf\" (UID: \"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf\") "
Jul 18 01:29:40.344513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.344489    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" (UID: "9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:40.445646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:40.445608    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:41.069471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:41.069438    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"
Jul 18 01:29:41.069471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:41.069447    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8" event={"ID":"9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf","Type":"ContainerDied","Data":"bc5609a86378aad64609c5dd9202bfad1c5a343357ce885ec26702379cacd8b4"}
Jul 18 01:29:41.069966 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:41.069527    2567 scope.go:117] "RemoveContainer" containerID="bb53509191a5de1879b91d2f7039804198250b268eda1130fc97c19e83038963"
Jul 18 01:29:41.090936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:41.090856    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"]
Jul 18 01:29:41.094282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:41.094241    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-lx9d8"]
Jul 18 01:29:42.920200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:42.920169    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" path="/var/lib/kubelet/pods/9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf/volumes"
Jul 18 01:29:45.970400 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:45.970367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:46.031973 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:46.031943    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:50.909235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:50.909188    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"]
Jul 18 01:29:50.909927 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:50.909581    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy" containerID="cri-o://332eb553438c9ca2a18f7d4df415ac8eae6d7b488c42894bc133d45031bccd8a" gracePeriod=30
Jul 18 01:29:50.913778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:50.913746    2567 scope.go:117] "RemoveContainer" containerID="0f5b8a191b82a2b0eaea19e776d9523d30db8aade2fa5d57ad746cc6d0cc1423"
Jul 18 01:29:50.929654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:50.929625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:51.155185 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.155150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:51.493516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.493423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:51.816888 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.816850    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"]
Jul 18 01:29:51.817320 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817303    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817320 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817320    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817336    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817341    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817353    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817360    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817430    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817445    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.817508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.817455    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bdd3d8a-dfac-46c2-a7f6-eb927bc8d5bf" containerName="revoke-keys"
Jul 18 01:29:51.820588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.820566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.822084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.822042    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:51.823369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.823347    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-feded4-openshift-default-dockercfg-pxgf9\""
Jul 18 01:29:51.833940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.833911    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"]
Jul 18 01:29:51.959225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959178    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-socket\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-token\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-certs\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-podinfo\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istiod-ca-cert\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959423    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jgpt\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-kube-api-access-6jgpt\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959472    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-credential-socket\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-data\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.959968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.959623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-envoy\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:51.994717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:51.994675    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:52.060556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060509    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-envoy\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-socket\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-token\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-certs\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-podinfo\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060969 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istiod-ca-cert\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060969 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jgpt\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-kube-api-access-6jgpt\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.060969 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-credential-socket\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.061153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.060978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-data\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.061153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.061045    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-socket\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.061153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.061106    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-certs\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.061336 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.061299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-data\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.061406 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.061346    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-credential-socket\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.061741 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.061718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istiod-ca-cert\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.063117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.063096    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-envoy\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.063614 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.063593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-podinfo\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.068518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.068447    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-token\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.068675 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.068652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jgpt\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-kube-api-access-6jgpt\") pod \"e2e-api-a-feded4-openshift-default-67978d9787-k7229\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") " pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.130440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.130405    2567 generic.go:356] "Generic (PLEG): container finished" podID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerID="bfddd8ef55a97219a2749aad3a828217ec363eb1e170dbde708dd7221c7ebd6e" exitCode=6
Jul 18 01:29:52.130588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.130480    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" event={"ID":"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b","Type":"ContainerDied","Data":"bfddd8ef55a97219a2749aad3a828217ec363eb1e170dbde708dd7221c7ebd6e"}
Jul 18 01:29:52.130588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.130569    2567 scope.go:117] "RemoveContainer" containerID="0f5b8a191b82a2b0eaea19e776d9523d30db8aade2fa5d57ad746cc6d0cc1423"
Jul 18 01:29:52.130857 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.130835    2567 scope.go:117] "RemoveContainer" containerID="bfddd8ef55a97219a2749aad3a828217ec363eb1e170dbde708dd7221c7ebd6e"
Jul 18 01:29:52.131141 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:52.131116    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf_odh-ai-gateway-infra(dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b"
Jul 18 01:29:52.133584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.133565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:52.287917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.287893    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"]
Jul 18 01:29:52.290061 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:29:52.290033    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdb232a_1c9b_48ba_9b2a_a0dc5105dc74.slice/crio-6902bdebcb5d0db64487c3b699ac90d339f7fa157491434fc153a15429f227a8 WatchSource:0}: Error finding container 6902bdebcb5d0db64487c3b699ac90d339f7fa157491434fc153a15429f227a8: Status 404 returned error can't find the container with id 6902bdebcb5d0db64487c3b699ac90d339f7fa157491434fc153a15429f227a8
Jul 18 01:29:52.292235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.292198    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:29:52.292345 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.292289    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:29:52.292345 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.292321    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:29:52.913726 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:52.913697    2567 scope.go:117] "RemoveContainer" containerID="dc6d0cbb52af5614958bb7d27780d07745bf809651fc9a68e81db2d3770cc795"
Jul 18 01:29:53.138402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.138366    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" event={"ID":"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74","Type":"ContainerStarted","Data":"10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"}
Jul 18 01:29:53.138881 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.138425    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" event={"ID":"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74","Type":"ContainerStarted","Data":"6902bdebcb5d0db64487c3b699ac90d339f7fa157491434fc153a15429f227a8"}
Jul 18 01:29:53.153977 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.153951    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"]
Jul 18 01:29:53.170937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.170884    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podStartSLOduration=2.170862592 podStartE2EDuration="2.170862592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:29:53.157188157 +0000 UTC m=+2972.684693117" watchObservedRunningTime="2026-07-18 01:29:53.170862592 +0000 UTC m=+2972.698367553"
Jul 18 01:29:53.296806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.296781    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:53.473952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.473861    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b-maas-api-service-ca\") pod \"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b\" (UID: \"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b\") "
Jul 18 01:29:53.474297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.474243    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" (UID: "dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:53.575128 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:53.575099    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:54.134431 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.134393    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:54.139381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.139354    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:54.146470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.146447    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"
Jul 18 01:29:54.146600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.146449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf" event={"ID":"dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b","Type":"ContainerDied","Data":"3c2d52330611880f0bde55aa2e15aa0462540122f30152c9f5f78f4eb2b3953b"}
Jul 18 01:29:54.146600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.146578    2567 scope.go:117] "RemoveContainer" containerID="bfddd8ef55a97219a2749aad3a828217ec363eb1e170dbde708dd7221c7ebd6e"
Jul 18 01:29:54.148578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.148557    2567 generic.go:356] "Generic (PLEG): container finished" podID="82cadb17-b743-469d-8da0-89a737101d4c" containerID="e044214707f947f2dafc1b62dc98b80b83a7f27a5e2ec8efad7791be5ad63e96" exitCode=6
Jul 18 01:29:54.148715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.148620    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" event={"ID":"82cadb17-b743-469d-8da0-89a737101d4c","Type":"ContainerDied","Data":"e044214707f947f2dafc1b62dc98b80b83a7f27a5e2ec8efad7791be5ad63e96"}
Jul 18 01:29:54.148891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.148866    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:54.149064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.149046    2567 scope.go:117] "RemoveContainer" containerID="e044214707f947f2dafc1b62dc98b80b83a7f27a5e2ec8efad7791be5ad63e96"
Jul 18 01:29:54.149314 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:29:54.149290    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-drw72_odh-ai-gateway-infra(82cadb17-b743-469d-8da0-89a737101d4c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" podUID="82cadb17-b743-469d-8da0-89a737101d4c"
Jul 18 01:29:54.150064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.150044    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:29:54.157490 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.157330    2567 scope.go:117] "RemoveContainer" containerID="dc6d0cbb52af5614958bb7d27780d07745bf809651fc9a68e81db2d3770cc795"
Jul 18 01:29:54.173578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.173546    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"]
Jul 18 01:29:54.177225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.177184    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-2jqbf"]
Jul 18 01:29:54.919431 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:54.919399    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" path="/var/lib/kubelet/pods/dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b/volumes"
Jul 18 01:29:55.157359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.157321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:55.214754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.214670    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"]
Jul 18 01:29:55.222460 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.222424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:55.375546 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.375519    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:55.447758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.447727    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:55.494135 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.494050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cadb17-b743-469d-8da0-89a737101d4c-maas-api-service-ca\") pod \"82cadb17-b743-469d-8da0-89a737101d4c\" (UID: \"82cadb17-b743-469d-8da0-89a737101d4c\") "
Jul 18 01:29:55.494466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.494444    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82cadb17-b743-469d-8da0-89a737101d4c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82cadb17-b743-469d-8da0-89a737101d4c" (UID: "82cadb17-b743-469d-8da0-89a737101d4c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:55.595245 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.595210    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cadb17-b743-469d-8da0-89a737101d4c-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:55.846085 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:55.846031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:56.012464 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.012431    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:56.113848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.113747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:56.168123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.167614    2567 generic.go:356] "Generic (PLEG): container finished" podID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerID="332eb553438c9ca2a18f7d4df415ac8eae6d7b488c42894bc133d45031bccd8a" exitCode=0
Jul 18 01:29:56.168123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.167652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" event={"ID":"eff0c3bc-7207-4f93-bca1-ca6e614f451b","Type":"ContainerDied","Data":"332eb553438c9ca2a18f7d4df415ac8eae6d7b488c42894bc133d45031bccd8a"}
Jul 18 01:29:56.168123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.167754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2" event={"ID":"eff0c3bc-7207-4f93-bca1-ca6e614f451b","Type":"ContainerDied","Data":"35b62ca9aaaac832285c71a1da8a5cb70acace09802e76931ba8ebd77a87b57f"}
Jul 18 01:29:56.168123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.167829    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35b62ca9aaaac832285c71a1da8a5cb70acace09802e76931ba8ebd77a87b57f"
Jul 18 01:29:56.168123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.167878    2567 scope.go:117] "RemoveContainer" containerID="cca0eadb75f6ae7fcb3be879e1ce37ba6d65201c6bcf1b4171695a6b1a5e4458"
Jul 18 01:29:56.170248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.169971    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"
Jul 18 01:29:56.170248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.170010    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72" event={"ID":"82cadb17-b743-469d-8da0-89a737101d4c","Type":"ContainerDied","Data":"bad5f6f7b64bad1ef7005b31ff8d4a1921da7a0cb19b773e758c96754150245e"}
Jul 18 01:29:56.172803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.172782    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:29:56.187059 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.187039    2567 scope.go:117] "RemoveContainer" containerID="e044214707f947f2dafc1b62dc98b80b83a7f27a5e2ec8efad7791be5ad63e96"
Jul 18 01:29:56.208787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.208754    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"]
Jul 18 01:29:56.210904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.210881    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-drw72"]
Jul 18 01:29:56.300779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.300740    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-certs\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.300779 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.300777    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istiod-ca-cert\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.300808    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lm9h\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-kube-api-access-4lm9h\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.300865    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-token\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.300888    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-data\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.300931    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-socket\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301082    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:56.301174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301102    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-envoy\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301245 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301171    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-credential-socket\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301245 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301203    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-podinfo\") pod \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\" (UID: \"eff0c3bc-7207-4f93-bca1-ca6e614f451b\") "
Jul 18 01:29:56.301382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301255    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:56.301382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301185    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:29:56.301503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301411    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:56.301809 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301704    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.301916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301884    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.301916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301901    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.301916 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.301914    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.302343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.302115    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-data" (OuterVolumeSpecName: "istio-data") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:56.303696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.303674    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:29:56.303811 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.303765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-kube-api-access-4lm9h" (OuterVolumeSpecName: "kube-api-access-4lm9h") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "kube-api-access-4lm9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:29:56.303811 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.303780    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-token" (OuterVolumeSpecName: "istio-token") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:29:56.303968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.303948    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eff0c3bc-7207-4f93-bca1-ca6e614f451b" (UID: "eff0c3bc-7207-4f93-bca1-ca6e614f451b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:29:56.403465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.403362    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.403465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.403396    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.403465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.403408    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.403465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.403419    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff0c3bc-7207-4f93-bca1-ca6e614f451b-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.403465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.403430    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lm9h\" (UniqueName: \"kubernetes.io/projected/eff0c3bc-7207-4f93-bca1-ca6e614f451b-kube-api-access-4lm9h\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:29:56.927638 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:56.927600    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82cadb17-b743-469d-8da0-89a737101d4c" path="/var/lib/kubelet/pods/82cadb17-b743-469d-8da0-89a737101d4c/volumes"
Jul 18 01:29:57.177061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:57.177024    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"
Jul 18 01:29:57.211878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:57.211783    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"]
Jul 18 01:29:57.217559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:57.217529    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8e3c6c-openshift-default-f764b89d7-ks2r2"]
Jul 18 01:29:58.919231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:58.919190    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" path="/var/lib/kubelet/pods/eff0c3bc-7207-4f93-bca1-ca6e614f451b/volumes"
Jul 18 01:29:59.587154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:59.587094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:29:59.809947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:29:59.809912    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:30:00.024331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.024209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:00.136315 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136259    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"]
Jul 18 01:30:00.136658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136645    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.136703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136660    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.136703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136673    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy"
Jul 18 01:30:00.136703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136679    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy"
Jul 18 01:30:00.136703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136685    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy"
Jul 18 01:30:00.136703 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136691    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136705    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136710    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136724    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136728    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136735    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136740    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136797    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136806    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136811    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136816    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff0c3bc-7207-4f93-bca1-ca6e614f451b" containerName="istio-proxy"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136822    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136828    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.136856 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.136833    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.139842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.139825    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"]
Jul 18 01:30:00.139963 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.139945    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"
Jul 18 01:30:00.140170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.140149    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.140170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.140165    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.140396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.140205    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.140396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.140213    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82cadb17-b743-469d-8da0-89a737101d4c" containerName="revoke-keys"
Jul 18 01:30:00.140396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.140316    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dff565e0-ee9c-4ee4-99ec-23c0f7cdae9b" containerName="revoke-keys"
Jul 18 01:30:00.142370 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.142351    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:30:00.143297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.143262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"
Jul 18 01:30:00.146724 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.146697    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"]
Jul 18 01:30:00.154422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.154257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"
Jul 18 01:30:00.162100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.162070    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"]
Jul 18 01:30:00.164461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.164437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"
Jul 18 01:30:00.309016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.308979    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"]
Jul 18 01:30:00.314415 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.314387    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.317109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.317046    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8775cd-openshift-default-dockercfg-zxl6g\""
Jul 18 01:30:00.330549 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.330511    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"]
Jul 18 01:30:00.332712 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:00.332683    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dd85de_a330_4e91_9f5b_201a480f581b.slice/crio-c12271418964d3e624cbdc954086c7d7cb47e3ebcdb9f4eb46d591f5060d21dd WatchSource:0}: Error finding container c12271418964d3e624cbdc954086c7d7cb47e3ebcdb9f4eb46d591f5060d21dd: Status 404 returned error can't find the container with id c12271418964d3e624cbdc954086c7d7cb47e3ebcdb9f4eb46d591f5060d21dd
Jul 18 01:30:00.333172 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.333135    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"]
Jul 18 01:30:00.334171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.334145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:00.437929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.437876    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-podinfo\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.437946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-socket\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.437972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-envoy\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.437989    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc8zk\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-kube-api-access-gc8zk\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.438064    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-token\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.438090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-credential-socket\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.438104    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-data\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438365 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.438122    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc16d383-9251-42b3-bd41-0fbafe289aae-istiod-ca-cert\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.438365 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.438155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-certs\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539470    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-socket\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-envoy\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc8zk\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-kube-api-access-gc8zk\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-token\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-credential-socket\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-data\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc16d383-9251-42b3-bd41-0fbafe289aae-istiod-ca-cert\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-certs\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.539864 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539804    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-podinfo\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.540382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.539932    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-socket\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.540382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.540227    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-credential-socket\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.540557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.540540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-certs\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.540695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.540672    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-data\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.540760 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.540722    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc16d383-9251-42b3-bd41-0fbafe289aae-istiod-ca-cert\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.542613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.542592    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-envoy\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.542829 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.542809    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-podinfo\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.548332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.548297    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc8zk\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-kube-api-access-gc8zk\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.548594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.548577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-token\") pod \"e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") " pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.552848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.552823    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:00.577716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.577691    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"]
Jul 18 01:30:00.633488 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.633458    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:00.798488 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:00.798458    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc16d383_9251_42b3_bd41_0fbafe289aae.slice/crio-68c138dd1f918dcb03abde1870461e8f068ba7fc55775aad6ea75f76db513e46 WatchSource:0}: Error finding container 68c138dd1f918dcb03abde1870461e8f068ba7fc55775aad6ea75f76db513e46: Status 404 returned error can't find the container with id 68c138dd1f918dcb03abde1870461e8f068ba7fc55775aad6ea75f76db513e46
Jul 18 01:30:00.801108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.801073    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:00.801184 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.801154    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:00.801221 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.801192    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:00.802367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:00.802345    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"]
Jul 18 01:30:01.204258 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.204152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb" event={"ID":"1f6c655b-68a4-43b9-95e5-0579f579e9f6","Type":"ContainerStarted","Data":"00101ca25c4c46f6eed9a25a68f6377267e30b8c79d2cd6dcf08d1fe9f4fbd12"}
Jul 18 01:30:01.204258 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.204215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb" event={"ID":"1f6c655b-68a4-43b9-95e5-0579f579e9f6","Type":"ContainerStarted","Data":"2b4aed39f14ba2287f1e8d4853a96925dec9012ea4e0b3f68106dc653c0449fa"}
Jul 18 01:30:01.206343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.206308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" event={"ID":"bc16d383-9251-42b3-bd41-0fbafe289aae","Type":"ContainerStarted","Data":"9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"}
Jul 18 01:30:01.206485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.206358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" event={"ID":"bc16d383-9251-42b3-bd41-0fbafe289aae","Type":"ContainerStarted","Data":"68c138dd1f918dcb03abde1870461e8f068ba7fc55775aad6ea75f76db513e46"}
Jul 18 01:30:01.208048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.208023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerStarted","Data":"981673ae759fdc4058d34bfb76a89b81a8ccdf44abef204bee3f14c0287ff7c3"}
Jul 18 01:30:01.208335 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.208310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerStarted","Data":"c12271418964d3e624cbdc954086c7d7cb47e3ebcdb9f4eb46d591f5060d21dd"}
Jul 18 01:30:01.218826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.218773    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb" podStartSLOduration=1.218756776 podStartE2EDuration="1.218756776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:01.218344004 +0000 UTC m=+2980.745848965" watchObservedRunningTime="2026-07-18 01:30:01.218756776 +0000 UTC m=+2980.746261737"
Jul 18 01:30:01.234565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.234503    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" podStartSLOduration=1.234487044 podStartE2EDuration="1.234487044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:01.232549288 +0000 UTC m=+2980.760054249" watchObservedRunningTime="2026-07-18 01:30:01.234487044 +0000 UTC m=+2980.761992004"
Jul 18 01:30:01.255296 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.255212    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" podStartSLOduration=1.2551924030000001 podStartE2EDuration="1.255192403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:01.254476703 +0000 UTC m=+2980.781981665" watchObservedRunningTime="2026-07-18 01:30:01.255192403 +0000 UTC m=+2980.782697365"
Jul 18 01:30:01.353973 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.353933    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"]
Jul 18 01:30:01.360631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.360571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:01.363884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.363858    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:30:01.371996 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.370035    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"]
Jul 18 01:30:01.448662 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.448623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad5a3a9-e46e-49b9-9c83-692e33c00422-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh\" (UID: \"3ad5a3a9-e46e-49b9-9c83-692e33c00422\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:01.550048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.549951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad5a3a9-e46e-49b9-9c83-692e33c00422-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh\" (UID: \"3ad5a3a9-e46e-49b9-9c83-692e33c00422\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:01.550774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.550738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad5a3a9-e46e-49b9-9c83-692e33c00422-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh\" (UID: \"3ad5a3a9-e46e-49b9-9c83-692e33c00422\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:01.634351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.634305    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:01.639614 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.639582    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:01.698899 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.698862    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:01.883395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:01.875254    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"]
Jul 18 01:30:01.893125 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:01.893082    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad5a3a9_e46e_49b9_9c83_692e33c00422.slice/crio-4d3eaed1d4b2147856eab3b386e92a316d877c5d9247f00a1ce323ec7b034537 WatchSource:0}: Error finding container 4d3eaed1d4b2147856eab3b386e92a316d877c5d9247f00a1ce323ec7b034537: Status 404 returned error can't find the container with id 4d3eaed1d4b2147856eab3b386e92a316d877c5d9247f00a1ce323ec7b034537
Jul 18 01:30:02.214289 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.214233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" event={"ID":"3ad5a3a9-e46e-49b9-9c83-692e33c00422","Type":"ContainerStarted","Data":"4d3eaed1d4b2147856eab3b386e92a316d877c5d9247f00a1ce323ec7b034537"}
Jul 18 01:30:02.215702 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.215673    2567 generic.go:356] "Generic (PLEG): container finished" podID="40dd85de-a330-4e91-9f5b-201a480f581b" containerID="981673ae759fdc4058d34bfb76a89b81a8ccdf44abef204bee3f14c0287ff7c3" exitCode=7
Jul 18 01:30:02.215848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.215761    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerDied","Data":"981673ae759fdc4058d34bfb76a89b81a8ccdf44abef204bee3f14c0287ff7c3"}
Jul 18 01:30:02.215932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.215919    2567 scope.go:117] "RemoveContainer" containerID="981673ae759fdc4058d34bfb76a89b81a8ccdf44abef204bee3f14c0287ff7c3"
Jul 18 01:30:02.217259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.217236    2567 generic.go:356] "Generic (PLEG): container finished" podID="1f6c655b-68a4-43b9-95e5-0579f579e9f6" containerID="00101ca25c4c46f6eed9a25a68f6377267e30b8c79d2cd6dcf08d1fe9f4fbd12" exitCode=0
Jul 18 01:30:02.217406 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.217369    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb" event={"ID":"1f6c655b-68a4-43b9-95e5-0579f579e9f6","Type":"ContainerDied","Data":"00101ca25c4c46f6eed9a25a68f6377267e30b8c79d2cd6dcf08d1fe9f4fbd12"}
Jul 18 01:30:02.217589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.217569    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:02.218624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:02.218607    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:03.166192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.166144    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:03.224658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.224625    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerID="a2f4f2bb6287179b0395ea551759df08a48c2b7c80ad16f81c72240db3b664c7" exitCode=6
Jul 18 01:30:03.225122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.224719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" event={"ID":"3ad5a3a9-e46e-49b9-9c83-692e33c00422","Type":"ContainerDied","Data":"a2f4f2bb6287179b0395ea551759df08a48c2b7c80ad16f81c72240db3b664c7"}
Jul 18 01:30:03.225122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.224938    2567 scope.go:117] "RemoveContainer" containerID="a2f4f2bb6287179b0395ea551759df08a48c2b7c80ad16f81c72240db3b664c7"
Jul 18 01:30:03.226762 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.226741    2567 generic.go:356] "Generic (PLEG): container finished" podID="40dd85de-a330-4e91-9f5b-201a480f581b" containerID="62d89ccae47610ea6158d7aaa50589b10d78eade705d72f39e172dd1b70a3890" exitCode=7
Jul 18 01:30:03.226879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.226808    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerDied","Data":"62d89ccae47610ea6158d7aaa50589b10d78eade705d72f39e172dd1b70a3890"}
Jul 18 01:30:03.226879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.226842    2567 scope.go:117] "RemoveContainer" containerID="981673ae759fdc4058d34bfb76a89b81a8ccdf44abef204bee3f14c0287ff7c3"
Jul 18 01:30:03.227013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.226996    2567 scope.go:117] "RemoveContainer" containerID="62d89ccae47610ea6158d7aaa50589b10d78eade705d72f39e172dd1b70a3890"
Jul 18 01:30:03.227398 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:03.227212    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db_odh-ai-gateway-infra(40dd85de-a330-4e91-9f5b-201a480f581b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" podUID="40dd85de-a330-4e91-9f5b-201a480f581b"
Jul 18 01:30:03.243388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.243336    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:03.418018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.417926    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"
Jul 18 01:30:03.431434 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.431406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:03.537867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.537834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:03.920671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.920636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:03.994611 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:03.994573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:04.233798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.233760    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb"
Jul 18 01:30:04.233798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.233782    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738970-mdwnb" event={"ID":"1f6c655b-68a4-43b9-95e5-0579f579e9f6","Type":"ContainerDied","Data":"2b4aed39f14ba2287f1e8d4853a96925dec9012ea4e0b3f68106dc653c0449fa"}
Jul 18 01:30:04.234286 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.233871    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b4aed39f14ba2287f1e8d4853a96925dec9012ea4e0b3f68106dc653c0449fa"
Jul 18 01:30:04.235817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.235786    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerID="ef72304d816327d9631ce6efbe3c50028b20fabd0f287e202616c653523f4d46" exitCode=6
Jul 18 01:30:04.235956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.235859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" event={"ID":"3ad5a3a9-e46e-49b9-9c83-692e33c00422","Type":"ContainerDied","Data":"ef72304d816327d9631ce6efbe3c50028b20fabd0f287e202616c653523f4d46"}
Jul 18 01:30:04.235956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.235919    2567 scope.go:117] "RemoveContainer" containerID="a2f4f2bb6287179b0395ea551759df08a48c2b7c80ad16f81c72240db3b664c7"
Jul 18 01:30:04.236075 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.236046    2567 scope.go:117] "RemoveContainer" containerID="ef72304d816327d9631ce6efbe3c50028b20fabd0f287e202616c653523f4d46"
Jul 18 01:30:04.236350 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:04.236326    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh_odh-ai-gateway-infra(3ad5a3a9-e46e-49b9-9c83-692e33c00422)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422"
Jul 18 01:30:04.237591 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:04.237575    2567 scope.go:117] "RemoveContainer" containerID="62d89ccae47610ea6158d7aaa50589b10d78eade705d72f39e172dd1b70a3890"
Jul 18 01:30:04.237778 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:04.237749    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db_odh-ai-gateway-infra(40dd85de-a330-4e91-9f5b-201a480f581b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" podUID="40dd85de-a330-4e91-9f5b-201a480f581b"
Jul 18 01:30:05.244764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:05.244737    2567 scope.go:117] "RemoveContainer" containerID="ef72304d816327d9631ce6efbe3c50028b20fabd0f287e202616c653523f4d46"
Jul 18 01:30:05.245221 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:05.245001    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh_odh-ai-gateway-infra(3ad5a3a9-e46e-49b9-9c83-692e33c00422)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422"
Jul 18 01:30:07.693208 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:07.693171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:08.271466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:08.271351    2567 generic.go:356] "Generic (PLEG): container finished" podID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerID="10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d" exitCode=0
Jul 18 01:30:08.271694 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:08.271532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" event={"ID":"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74","Type":"ContainerDied","Data":"10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"}
Jul 18 01:30:08.272038 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:08.272017    2567 scope.go:117] "RemoveContainer" containerID="10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"
Jul 18 01:30:08.273686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:08.273634    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:08.273811 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:08.273759    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:08.273877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:08.273826    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:09.149745 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:09.149708    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:09.284977 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:09.284939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" event={"ID":"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74","Type":"ContainerStarted","Data":"d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b"}
Jul 18 01:30:10.134556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:10.134516    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:10.135553 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:10.135521    2567 patch_prober.go:28] interesting pod/e2e-api-a-feded4-openshift-default-67978d9787-k7229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Jul 18 01:30:10.135693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:10.135572    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Jul 18 01:30:11.134567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:11.134527    2567 patch_prober.go:28] interesting pod/e2e-api-a-feded4-openshift-default-67978d9787-k7229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Jul 18 01:30:11.134962 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:11.134583    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Jul 18 01:30:12.135093 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:12.135054    2567 patch_prober.go:28] interesting pod/e2e-api-a-feded4-openshift-default-67978d9787-k7229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Jul 18 01:30:12.135487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:12.135133    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Jul 18 01:30:13.134079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:13.134038    2567 patch_prober.go:28] interesting pod/e2e-api-a-feded4-openshift-default-67978d9787-k7229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Jul 18 01:30:13.134297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:13.134093    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Jul 18 01:30:14.138720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:14.138679    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:14.139232 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:14.139007    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:14.139913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:14.139888    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:15.817716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.817679    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"]
Jul 18 01:30:15.818145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.818098    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6c655b-68a4-43b9-95e5-0579f579e9f6" containerName="cleanup"
Jul 18 01:30:15.818145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.818111    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6c655b-68a4-43b9-95e5-0579f579e9f6" containerName="cleanup"
Jul 18 01:30:15.818231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.818184    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f6c655b-68a4-43b9-95e5-0579f579e9f6" containerName="cleanup"
Jul 18 01:30:15.821197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.821180    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:15.828212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.828185    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"]
Jul 18 01:30:15.907951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:15.907916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de796c2-f9ae-426a-a7eb-6dae98935ec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-feded4-gxdkz\" (UID: \"9de796c2-f9ae-426a-a7eb-6dae98935ec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:16.009079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.009033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de796c2-f9ae-426a-a7eb-6dae98935ec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-feded4-gxdkz\" (UID: \"9de796c2-f9ae-426a-a7eb-6dae98935ec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:16.009695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.009672    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de796c2-f9ae-426a-a7eb-6dae98935ec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-feded4-gxdkz\" (UID: \"9de796c2-f9ae-426a-a7eb-6dae98935ec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:16.134830 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.134733    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:16.279288 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.279248    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"]
Jul 18 01:30:16.281601 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:16.281558    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de796c2_f9ae_426a_a7eb_6dae98935ec8.slice/crio-55c54c82d58ccb38ef1e3144ad3c090278e4b298042e3b6df58b31a56c050ccd WatchSource:0}: Error finding container 55c54c82d58ccb38ef1e3144ad3c090278e4b298042e3b6df58b31a56c050ccd: Status 404 returned error can't find the container with id 55c54c82d58ccb38ef1e3144ad3c090278e4b298042e3b6df58b31a56c050ccd
Jul 18 01:30:16.336409 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.336381    2567 generic.go:356] "Generic (PLEG): container finished" podID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerID="9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b" exitCode=0
Jul 18 01:30:16.336594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.336459    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" event={"ID":"bc16d383-9251-42b3-bd41-0fbafe289aae","Type":"ContainerDied","Data":"9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"}
Jul 18 01:30:16.336976 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.336949    2567 scope.go:117] "RemoveContainer" containerID="9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"
Jul 18 01:30:16.337788 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.337762    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz" event={"ID":"9de796c2-f9ae-426a-a7eb-6dae98935ec8","Type":"ContainerStarted","Data":"55c54c82d58ccb38ef1e3144ad3c090278e4b298042e3b6df58b31a56c050ccd"}
Jul 18 01:30:16.338026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.337999    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:16.338104 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.338086    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:16.338169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.338134    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:30:16.419771 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.419736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:16.522201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:16.522168    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:17.218052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.217995    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:17.345725 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.345690    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" event={"ID":"bc16d383-9251-42b3-bd41-0fbafe289aae","Type":"ContainerStarted","Data":"3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2"}
Jul 18 01:30:17.347234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.347210    2567 generic.go:356] "Generic (PLEG): container finished" podID="9de796c2-f9ae-426a-a7eb-6dae98935ec8" containerID="0574d6f14a78b0c71dcb8031139850cdc7dad15db18063532722fc3c1dc1ec2f" exitCode=0
Jul 18 01:30:17.347433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.347301    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz" event={"ID":"9de796c2-f9ae-426a-a7eb-6dae98935ec8","Type":"ContainerDied","Data":"0574d6f14a78b0c71dcb8031139850cdc7dad15db18063532722fc3c1dc1ec2f"}
Jul 18 01:30:17.634592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.634492    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:17.635882 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.635853    2567 patch_prober.go:28] interesting pod/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 18 01:30:17.636008 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.635903    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 18 01:30:17.813764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:17.813722    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:30:18.537024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:18.537001    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:18.633229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:18.633136    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de796c2-f9ae-426a-a7eb-6dae98935ec8-maas-api-service-ca\") pod \"9de796c2-f9ae-426a-a7eb-6dae98935ec8\" (UID: \"9de796c2-f9ae-426a-a7eb-6dae98935ec8\") "
Jul 18 01:30:18.633516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:18.633493    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9de796c2-f9ae-426a-a7eb-6dae98935ec8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9de796c2-f9ae-426a-a7eb-6dae98935ec8" (UID: "9de796c2-f9ae-426a-a7eb-6dae98935ec8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:18.633827 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:18.633808    2567 patch_prober.go:28] interesting pod/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 18 01:30:18.633881 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:18.633857    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 18 01:30:18.734614 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:18.734578    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9de796c2-f9ae-426a-a7eb-6dae98935ec8-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:19.199537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.199448    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"]
Jul 18 01:30:19.199871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.199859    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de796c2-f9ae-426a-a7eb-6dae98935ec8" containerName="revoke-keys"
Jul 18 01:30:19.199919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.199873    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de796c2-f9ae-426a-a7eb-6dae98935ec8" containerName="revoke-keys"
Jul 18 01:30:19.199955 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.199938    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de796c2-f9ae-426a-a7eb-6dae98935ec8" containerName="revoke-keys"
Jul 18 01:30:19.203401 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.203386    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:19.211047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.211021    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"]
Jul 18 01:30:19.241224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.241177    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7\" (UID: \"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:19.342217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.342172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7\" (UID: \"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:19.342890 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.342867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7\" (UID: \"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:19.370068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.370029    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz"
Jul 18 01:30:19.370525 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.370499    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-feded4-gxdkz" event={"ID":"9de796c2-f9ae-426a-a7eb-6dae98935ec8","Type":"ContainerDied","Data":"55c54c82d58ccb38ef1e3144ad3c090278e4b298042e3b6df58b31a56c050ccd"}
Jul 18 01:30:19.370661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.370548    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55c54c82d58ccb38ef1e3144ad3c090278e4b298042e3b6df58b31a56c050ccd"
Jul 18 01:30:19.515913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.515823    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:19.634554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.634175    2567 patch_prober.go:28] interesting pod/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 18 01:30:19.634554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.634222    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 18 01:30:19.709117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.709059    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"]
Jul 18 01:30:19.712464 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:19.712436    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85a4d80_b9cf_41f2_a023_09a1f75cb4c4.slice/crio-c9f48ab0c08a70fd164fd5cc13e6e9ae31ec1dc8fc77e56d65cea861932b9e0f WatchSource:0}: Error finding container c9f48ab0c08a70fd164fd5cc13e6e9ae31ec1dc8fc77e56d65cea861932b9e0f: Status 404 returned error can't find the container with id c9f48ab0c08a70fd164fd5cc13e6e9ae31ec1dc8fc77e56d65cea861932b9e0f
Jul 18 01:30:19.913672 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:19.913642    2567 scope.go:117] "RemoveContainer" containerID="62d89ccae47610ea6158d7aaa50589b10d78eade705d72f39e172dd1b70a3890"
Jul 18 01:30:20.382010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.381963    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerStarted","Data":"6f17192880126b69df39f317d54e1f49aebb274b46cbaacace138888f870323c"}
Jul 18 01:30:20.383666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.383626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerStarted","Data":"db7caa33235ebd5860d8c2afd0f6a1e4907f1a5e6a373bbeb414fb7ea808b1f7"}
Jul 18 01:30:20.383814 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.383687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerStarted","Data":"c9f48ab0c08a70fd164fd5cc13e6e9ae31ec1dc8fc77e56d65cea861932b9e0f"}
Jul 18 01:30:20.409308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.409235    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" podStartSLOduration=1.409220591 podStartE2EDuration="1.409220591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:20.408439976 +0000 UTC m=+2999.935944935" watchObservedRunningTime="2026-07-18 01:30:20.409220591 +0000 UTC m=+2999.936725550"
Jul 18 01:30:20.634337 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.634297    2567 patch_prober.go:28] interesting pod/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 18 01:30:20.634526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.634357    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 18 01:30:20.916228 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:20.916144    2567 scope.go:117] "RemoveContainer" containerID="ef72304d816327d9631ce6efbe3c50028b20fabd0f287e202616c653523f4d46"
Jul 18 01:30:21.109093 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.109067    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:30:21.114369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.114345    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:30:21.392181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.392152    2567 generic.go:356] "Generic (PLEG): container finished" podID="40dd85de-a330-4e91-9f5b-201a480f581b" containerID="6f17192880126b69df39f317d54e1f49aebb274b46cbaacace138888f870323c" exitCode=0
Jul 18 01:30:21.392419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.392231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerDied","Data":"6f17192880126b69df39f317d54e1f49aebb274b46cbaacace138888f870323c"}
Jul 18 01:30:21.392419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.392328    2567 scope.go:117] "RemoveContainer" containerID="62d89ccae47610ea6158d7aaa50589b10d78eade705d72f39e172dd1b70a3890"
Jul 18 01:30:21.393846 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.393816    2567 generic.go:356] "Generic (PLEG): container finished" podID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerID="db7caa33235ebd5860d8c2afd0f6a1e4907f1a5e6a373bbeb414fb7ea808b1f7" exitCode=6
Jul 18 01:30:21.393964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.393903    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerDied","Data":"db7caa33235ebd5860d8c2afd0f6a1e4907f1a5e6a373bbeb414fb7ea808b1f7"}
Jul 18 01:30:21.394078 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.394063    2567 scope.go:117] "RemoveContainer" containerID="db7caa33235ebd5860d8c2afd0f6a1e4907f1a5e6a373bbeb414fb7ea808b1f7"
Jul 18 01:30:21.396012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.395993    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" event={"ID":"3ad5a3a9-e46e-49b9-9c83-692e33c00422","Type":"ContainerStarted","Data":"1aa2bc9e191f15bd211e32023a553a6eb41f607e8f1d9aa91e8931af841f57dc"}
Jul 18 01:30:21.405594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.405564    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"]
Jul 18 01:30:21.433429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.433384    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" podStartSLOduration=20.433367376 podStartE2EDuration="20.433367376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:21.430431072 +0000 UTC m=+3000.957936032" watchObservedRunningTime="2026-07-18 01:30:21.433367376 +0000 UTC m=+3000.960872332"
Jul 18 01:30:21.638503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.638472    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:21.638786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.638759    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:21.639537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:21.639518    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:30:22.403552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.403515    2567 generic.go:356] "Generic (PLEG): container finished" podID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerID="1aa2bc9e191f15bd211e32023a553a6eb41f607e8f1d9aa91e8931af841f57dc" exitCode=6
Jul 18 01:30:22.404112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.403594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" event={"ID":"3ad5a3a9-e46e-49b9-9c83-692e33c00422","Type":"ContainerDied","Data":"1aa2bc9e191f15bd211e32023a553a6eb41f607e8f1d9aa91e8931af841f57dc"}
Jul 18 01:30:22.404112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.403677    2567 scope.go:117] "RemoveContainer" containerID="ef72304d816327d9631ce6efbe3c50028b20fabd0f287e202616c653523f4d46"
Jul 18 01:30:22.404112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.403997    2567 scope.go:117] "RemoveContainer" containerID="1aa2bc9e191f15bd211e32023a553a6eb41f607e8f1d9aa91e8931af841f57dc"
Jul 18 01:30:22.404303 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:22.404251    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh_odh-ai-gateway-infra(3ad5a3a9-e46e-49b9-9c83-692e33c00422)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422"
Jul 18 01:30:22.407349 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.407323    2567 generic.go:356] "Generic (PLEG): container finished" podID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerID="543d794a25ea0fc1611162840b540d1d7117d1ecea5f2beef70ec40ede8738f4" exitCode=6
Jul 18 01:30:22.407505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.407482    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerDied","Data":"543d794a25ea0fc1611162840b540d1d7117d1ecea5f2beef70ec40ede8738f4"}
Jul 18 01:30:22.407796 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.407775    2567 scope.go:117] "RemoveContainer" containerID="543d794a25ea0fc1611162840b540d1d7117d1ecea5f2beef70ec40ede8738f4"
Jul 18 01:30:22.408103 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:22.408079    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7_odh-ai-gateway-infra(a85a4d80-b9cf-41f2-a023-09a1f75cb4c4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4"
Jul 18 01:30:22.415462 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.415440    2567 scope.go:117] "RemoveContainer" containerID="db7caa33235ebd5860d8c2afd0f6a1e4907f1a5e6a373bbeb414fb7ea808b1f7"
Jul 18 01:30:22.440377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.440347    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"]
Jul 18 01:30:22.550860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:22.550827    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"
Jul 18 01:30:23.415710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.415680    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"
Jul 18 01:30:23.416107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.415699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db" event={"ID":"40dd85de-a330-4e91-9f5b-201a480f581b","Type":"ContainerDied","Data":"c12271418964d3e624cbdc954086c7d7cb47e3ebcdb9f4eb46d591f5060d21dd"}
Jul 18 01:30:23.416107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.415760    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c12271418964d3e624cbdc954086c7d7cb47e3ebcdb9f4eb46d591f5060d21dd"
Jul 18 01:30:23.418074 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.418047    2567 scope.go:117] "RemoveContainer" containerID="543d794a25ea0fc1611162840b540d1d7117d1ecea5f2beef70ec40ede8738f4"
Jul 18 01:30:23.418332 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:23.418306    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7_odh-ai-gateway-infra(a85a4d80-b9cf-41f2-a023-09a1f75cb4c4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4"
Jul 18 01:30:23.451548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.451511    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"]
Jul 18 01:30:23.459340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.459310    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-feded4-29738970-mt6db"]
Jul 18 01:30:23.574651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.574623    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:23.687191 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.687095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad5a3a9-e46e-49b9-9c83-692e33c00422-maas-api-service-ca\") pod \"3ad5a3a9-e46e-49b9-9c83-692e33c00422\" (UID: \"3ad5a3a9-e46e-49b9-9c83-692e33c00422\") "
Jul 18 01:30:23.687515 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.687492    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ad5a3a9-e46e-49b9-9c83-692e33c00422-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ad5a3a9-e46e-49b9-9c83-692e33c00422" (UID: "3ad5a3a9-e46e-49b9-9c83-692e33c00422"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:23.787973 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:23.787933    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad5a3a9-e46e-49b9-9c83-692e33c00422-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:24.424668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.424638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh" event={"ID":"3ad5a3a9-e46e-49b9-9c83-692e33c00422","Type":"ContainerDied","Data":"4d3eaed1d4b2147856eab3b386e92a316d877c5d9247f00a1ce323ec7b034537"}
Jul 18 01:30:24.424668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.424667    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"
Jul 18 01:30:24.425188 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.424698    2567 scope.go:117] "RemoveContainer" containerID="1aa2bc9e191f15bd211e32023a553a6eb41f607e8f1d9aa91e8931af841f57dc"
Jul 18 01:30:24.445886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.445855    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"]
Jul 18 01:30:24.448595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.448566    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-zv2dh"]
Jul 18 01:30:24.664767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.664733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:24.765461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.765380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:24.919540 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.919506    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" path="/var/lib/kubelet/pods/3ad5a3a9-e46e-49b9-9c83-692e33c00422/volumes"
Jul 18 01:30:24.919816 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:24.919803    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" path="/var/lib/kubelet/pods/40dd85de-a330-4e91-9f5b-201a480f581b/volumes"
Jul 18 01:30:26.418491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.418448    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"]
Jul 18 01:30:26.419170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419147    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419174    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419191    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419199    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419215    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419223    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419237    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419245    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419302    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419311    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419434    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419450    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419462    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419471    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:26.419646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.419482    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:26.423752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.423727    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:26.432801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.432775    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"]
Jul 18 01:30:26.450449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.450411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:26.513594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.513558    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41408393-2234-427c-97bf-f5929d6a92ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6\" (UID: \"41408393-2234-427c-97bf-f5929d6a92ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:26.548879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.548847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:26.614878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.614836    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41408393-2234-427c-97bf-f5929d6a92ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6\" (UID: \"41408393-2234-427c-97bf-f5929d6a92ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:26.615584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.615556    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41408393-2234-427c-97bf-f5929d6a92ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6\" (UID: \"41408393-2234-427c-97bf-f5929d6a92ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:26.701884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.701786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:26.735332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.735231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:26.914433 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:26.914394    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41408393_2234_427c_97bf_f5929d6a92ed.slice/crio-049a76c022a957c72d3447ead767a46514db3449873d04187f6d3ba1f1c18cf8 WatchSource:0}: Error finding container 049a76c022a957c72d3447ead767a46514db3449873d04187f6d3ba1f1c18cf8: Status 404 returned error can't find the container with id 049a76c022a957c72d3447ead767a46514db3449873d04187f6d3ba1f1c18cf8
Jul 18 01:30:26.919683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:26.919659    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"]
Jul 18 01:30:27.446591 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:27.446499    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerStarted","Data":"990633968cf89c69a6f74ac78a195267a500c58cd5bb184348c5d3ff83569bf8"}
Jul 18 01:30:27.446591 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:27.446554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerStarted","Data":"049a76c022a957c72d3447ead767a46514db3449873d04187f6d3ba1f1c18cf8"}
Jul 18 01:30:27.462673 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:27.462623    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" podStartSLOduration=1.4626063679999999 podStartE2EDuration="1.462606368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:27.460001842 +0000 UTC m=+3006.987506803" watchObservedRunningTime="2026-07-18 01:30:27.462606368 +0000 UTC m=+3006.990111328"
Jul 18 01:30:28.453234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:28.453186    2567 generic.go:356] "Generic (PLEG): container finished" podID="41408393-2234-427c-97bf-f5929d6a92ed" containerID="990633968cf89c69a6f74ac78a195267a500c58cd5bb184348c5d3ff83569bf8" exitCode=6
Jul 18 01:30:28.453708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:28.453320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerDied","Data":"990633968cf89c69a6f74ac78a195267a500c58cd5bb184348c5d3ff83569bf8"}
Jul 18 01:30:28.453708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:28.453572    2567 scope.go:117] "RemoveContainer" containerID="990633968cf89c69a6f74ac78a195267a500c58cd5bb184348c5d3ff83569bf8"
Jul 18 01:30:29.462610 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:29.462521    2567 generic.go:356] "Generic (PLEG): container finished" podID="41408393-2234-427c-97bf-f5929d6a92ed" containerID="35be8821d0524343852478ac88ebe7ed857cd9dfba03ba79829728b83a452438" exitCode=6
Jul 18 01:30:29.463026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:29.462607    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerDied","Data":"35be8821d0524343852478ac88ebe7ed857cd9dfba03ba79829728b83a452438"}
Jul 18 01:30:29.463026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:29.462664    2567 scope.go:117] "RemoveContainer" containerID="990633968cf89c69a6f74ac78a195267a500c58cd5bb184348c5d3ff83569bf8"
Jul 18 01:30:29.463026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:29.462851    2567 scope.go:117] "RemoveContainer" containerID="35be8821d0524343852478ac88ebe7ed857cd9dfba03ba79829728b83a452438"
Jul 18 01:30:29.463160 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:29.463106    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6_odh-ai-gateway-infra(41408393-2234-427c-97bf-f5929d6a92ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" podUID="41408393-2234-427c-97bf-f5929d6a92ed"
Jul 18 01:30:30.401426 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:30.401395    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:30:30.470347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:30.470316    2567 scope.go:117] "RemoveContainer" containerID="35be8821d0524343852478ac88ebe7ed857cd9dfba03ba79829728b83a452438"
Jul 18 01:30:30.470734 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:30.470521    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6_odh-ai-gateway-infra(41408393-2234-427c-97bf-f5929d6a92ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" podUID="41408393-2234-427c-97bf-f5929d6a92ed"
Jul 18 01:30:30.915027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:30.914990    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:33.819618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.819571    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"]
Jul 18 01:30:33.820158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.820137    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:33.820257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.820162    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dd85de-a330-4e91-9f5b-201a480f581b" containerName="cleanup"
Jul 18 01:30:33.820350 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.820307    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad5a3a9-e46e-49b9-9c83-692e33c00422" containerName="revoke-keys"
Jul 18 01:30:33.824851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.824828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:33.830198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.830169    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"]
Jul 18 01:30:33.894920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.894861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3067e99-a980-424d-a1f0-98b176a36655-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-klwpk\" (UID: \"b3067e99-a980-424d-a1f0-98b176a36655\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:33.913063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.913035    2567 scope.go:117] "RemoveContainer" containerID="543d794a25ea0fc1611162840b540d1d7117d1ecea5f2beef70ec40ede8738f4"
Jul 18 01:30:33.996368 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.996327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3067e99-a980-424d-a1f0-98b176a36655-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-klwpk\" (UID: \"b3067e99-a980-424d-a1f0-98b176a36655\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:33.997034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:33.997010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3067e99-a980-424d-a1f0-98b176a36655-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-klwpk\" (UID: \"b3067e99-a980-424d-a1f0-98b176a36655\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:34.137391 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:34.137355    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:34.279443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:34.279417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"]
Jul 18 01:30:34.282765 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:34.282736    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3067e99_a980_424d_a1f0_98b176a36655.slice/crio-bed4783aa8de34960de2c3c5e675863b6e14d15edcf020947e6f2abc05a252f6 WatchSource:0}: Error finding container bed4783aa8de34960de2c3c5e675863b6e14d15edcf020947e6f2abc05a252f6: Status 404 returned error can't find the container with id bed4783aa8de34960de2c3c5e675863b6e14d15edcf020947e6f2abc05a252f6
Jul 18 01:30:34.496781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:34.496742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerStarted","Data":"64176a5852ef5535b130f38a161d2e2ca633b57caf15b5840a2c9fe2c78dc826"}
Jul 18 01:30:34.497949 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:34.497922    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" event={"ID":"b3067e99-a980-424d-a1f0-98b176a36655","Type":"ContainerStarted","Data":"bed4783aa8de34960de2c3c5e675863b6e14d15edcf020947e6f2abc05a252f6"}
Jul 18 01:30:35.505027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.504996    2567 generic.go:356] "Generic (PLEG): container finished" podID="b3067e99-a980-424d-a1f0-98b176a36655" containerID="e51c0f495dcab96e1fbfba16b3322919447c3f5b421987454a8b465c05b2d495" exitCode=6
Jul 18 01:30:35.505523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.505088    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" event={"ID":"b3067e99-a980-424d-a1f0-98b176a36655","Type":"ContainerDied","Data":"e51c0f495dcab96e1fbfba16b3322919447c3f5b421987454a8b465c05b2d495"}
Jul 18 01:30:35.505523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.505450    2567 scope.go:117] "RemoveContainer" containerID="e51c0f495dcab96e1fbfba16b3322919447c3f5b421987454a8b465c05b2d495"
Jul 18 01:30:35.507122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.507099    2567 generic.go:356] "Generic (PLEG): container finished" podID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerID="64176a5852ef5535b130f38a161d2e2ca633b57caf15b5840a2c9fe2c78dc826" exitCode=6
Jul 18 01:30:35.507251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.507138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerDied","Data":"64176a5852ef5535b130f38a161d2e2ca633b57caf15b5840a2c9fe2c78dc826"}
Jul 18 01:30:35.507251 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.507194    2567 scope.go:117] "RemoveContainer" containerID="543d794a25ea0fc1611162840b540d1d7117d1ecea5f2beef70ec40ede8738f4"
Jul 18 01:30:35.507576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.507527    2567 scope.go:117] "RemoveContainer" containerID="64176a5852ef5535b130f38a161d2e2ca633b57caf15b5840a2c9fe2c78dc826"
Jul 18 01:30:35.507758 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:35.507708    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7_odh-ai-gateway-infra(a85a4d80-b9cf-41f2-a023-09a1f75cb4c4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4"
Jul 18 01:30:35.521223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:35.521192    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"]
Jul 18 01:30:36.514891 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.514856    2567 generic.go:356] "Generic (PLEG): container finished" podID="b3067e99-a980-424d-a1f0-98b176a36655" containerID="269aaa44358f95daf11c6b44b1dd24e80261cc19631db43e2a863c8a30be0a37" exitCode=6
Jul 18 01:30:36.515334 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.514939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" event={"ID":"b3067e99-a980-424d-a1f0-98b176a36655","Type":"ContainerDied","Data":"269aaa44358f95daf11c6b44b1dd24e80261cc19631db43e2a863c8a30be0a37"}
Jul 18 01:30:36.515334 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.514994    2567 scope.go:117] "RemoveContainer" containerID="e51c0f495dcab96e1fbfba16b3322919447c3f5b421987454a8b465c05b2d495"
Jul 18 01:30:36.515334 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.515153    2567 scope.go:117] "RemoveContainer" containerID="269aaa44358f95daf11c6b44b1dd24e80261cc19631db43e2a863c8a30be0a37"
Jul 18 01:30:36.515501 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:36.515474    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-klwpk_odh-ai-gateway-infra(b3067e99-a980-424d-a1f0-98b176a36655)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" podUID="b3067e99-a980-424d-a1f0-98b176a36655"
Jul 18 01:30:36.665898 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.665869    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:36.761492 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.761459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:36.822737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.822707    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4-maas-api-service-ca\") pod \"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4\" (UID: \"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4\") "
Jul 18 01:30:36.823120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.823097    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" (UID: "a85a4d80-b9cf-41f2-a023-09a1f75cb4c4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:36.861389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.861356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:36.923475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:36.923442    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:37.528039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:37.527984    2567 scope.go:117] "RemoveContainer" containerID="269aaa44358f95daf11c6b44b1dd24e80261cc19631db43e2a863c8a30be0a37"
Jul 18 01:30:37.528540 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:37.528256    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-klwpk_odh-ai-gateway-infra(b3067e99-a980-424d-a1f0-98b176a36655)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" podUID="b3067e99-a980-424d-a1f0-98b176a36655"
Jul 18 01:30:37.529034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:37.529011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7" event={"ID":"a85a4d80-b9cf-41f2-a023-09a1f75cb4c4","Type":"ContainerDied","Data":"c9f48ab0c08a70fd164fd5cc13e6e9ae31ec1dc8fc77e56d65cea861932b9e0f"}
Jul 18 01:30:37.529150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:37.529041    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"
Jul 18 01:30:37.529150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:37.529068    2567 scope.go:117] "RemoveContainer" containerID="64176a5852ef5535b130f38a161d2e2ca633b57caf15b5840a2c9fe2c78dc826"
Jul 18 01:30:37.554236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:37.554201    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"]
Jul 18 01:30:37.556536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:37.556503    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-bpwn7"]
Jul 18 01:30:38.919170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:38.919133    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" path="/var/lib/kubelet/pods/a85a4d80-b9cf-41f2-a023-09a1f75cb4c4/volumes"
Jul 18 01:30:41.605172 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:41.605137    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"]
Jul 18 01:30:41.605677 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:41.605416    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy" containerID="cri-o://d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b" gracePeriod=30
Jul 18 01:30:41.653023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:41.652987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:41.874601 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:41.874511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:42.003010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.002963    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"]
Jul 18 01:30:42.003381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003369    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003383    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003413    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003419    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003425    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003443 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003431    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003490    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003497    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.003596 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.003505    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85a4d80-b9cf-41f2-a023-09a1f75cb4c4" containerName="revoke-keys"
Jul 18 01:30:42.007708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.007690    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:42.012707 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.012674    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"]
Jul 18 01:30:42.174796 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.174696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/645ebe55-aec8-4145-bac5-a4c684761e7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8775cd-96bnr\" (UID: \"645ebe55-aec8-4145-bac5-a4c684761e7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:42.275455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.275409    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/645ebe55-aec8-4145-bac5-a4c684761e7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8775cd-96bnr\" (UID: \"645ebe55-aec8-4145-bac5-a4c684761e7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:42.276100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.276074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/645ebe55-aec8-4145-bac5-a4c684761e7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8775cd-96bnr\" (UID: \"645ebe55-aec8-4145-bac5-a4c684761e7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:42.320339 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.320305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:42.461832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.461792    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"]
Jul 18 01:30:42.463999 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:30:42.463967    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645ebe55_aec8_4145_bac5_a4c684761e7a.slice/crio-87ee425e6f1b81c71f95e1e13e3a7195198f1b5575345b915fe49ea2b4a8092c WatchSource:0}: Error finding container 87ee425e6f1b81c71f95e1e13e3a7195198f1b5575345b915fe49ea2b4a8092c: Status 404 returned error can't find the container with id 87ee425e6f1b81c71f95e1e13e3a7195198f1b5575345b915fe49ea2b4a8092c
Jul 18 01:30:42.540918 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.540884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:42.560884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.560850    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr" event={"ID":"645ebe55-aec8-4145-bac5-a4c684761e7a","Type":"ContainerStarted","Data":"87ee425e6f1b81c71f95e1e13e3a7195198f1b5575345b915fe49ea2b4a8092c"}
Jul 18 01:30:42.641718 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.641683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:42.913427 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:42.913396    2567 scope.go:117] "RemoveContainer" containerID="35be8821d0524343852478ac88ebe7ed857cd9dfba03ba79829728b83a452438"
Jul 18 01:30:43.568748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:43.568708    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerStarted","Data":"ae7482af30e9914e3f53d7c09a80024bf89da279f5894c1cc96b3b5ba8f6ec4d"}
Jul 18 01:30:43.570323 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:43.570297    2567 generic.go:356] "Generic (PLEG): container finished" podID="645ebe55-aec8-4145-bac5-a4c684761e7a" containerID="2ce3447875bff34c677b33d3cd89eea99a19b41587b48f242c5ca63aac4845c6" exitCode=0
Jul 18 01:30:43.570460 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:43.570397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr" event={"ID":"645ebe55-aec8-4145-bac5-a4c684761e7a","Type":"ContainerDied","Data":"2ce3447875bff34c677b33d3cd89eea99a19b41587b48f242c5ca63aac4845c6"}
Jul 18 01:30:44.142082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.142048    2567 patch_prober.go:28] interesting pod/e2e-api-a-feded4-openshift-default-67978d9787-k7229 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 01:30:44.142546 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.142108    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 01:30:44.578238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.578207    2567 generic.go:356] "Generic (PLEG): container finished" podID="41408393-2234-427c-97bf-f5929d6a92ed" containerID="ae7482af30e9914e3f53d7c09a80024bf89da279f5894c1cc96b3b5ba8f6ec4d" exitCode=6
Jul 18 01:30:44.578444 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.578294    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerDied","Data":"ae7482af30e9914e3f53d7c09a80024bf89da279f5894c1cc96b3b5ba8f6ec4d"}
Jul 18 01:30:44.578444 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.578351    2567 scope.go:117] "RemoveContainer" containerID="35be8821d0524343852478ac88ebe7ed857cd9dfba03ba79829728b83a452438"
Jul 18 01:30:44.579044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.578732    2567 scope.go:117] "RemoveContainer" containerID="ae7482af30e9914e3f53d7c09a80024bf89da279f5894c1cc96b3b5ba8f6ec4d"
Jul 18 01:30:44.579044 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:44.579008    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6_odh-ai-gateway-infra(41408393-2234-427c-97bf-f5929d6a92ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" podUID="41408393-2234-427c-97bf-f5929d6a92ed"
Jul 18 01:30:44.594561 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.594533    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"]
Jul 18 01:30:44.725442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.725414    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:44.900468 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.900378    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/645ebe55-aec8-4145-bac5-a4c684761e7a-maas-api-service-ca\") pod \"645ebe55-aec8-4145-bac5-a4c684761e7a\" (UID: \"645ebe55-aec8-4145-bac5-a4c684761e7a\") "
Jul 18 01:30:44.900747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:44.900723    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/645ebe55-aec8-4145-bac5-a4c684761e7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "645ebe55-aec8-4145-bac5-a4c684761e7a" (UID: "645ebe55-aec8-4145-bac5-a4c684761e7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:45.001733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.001694    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/645ebe55-aec8-4145-bac5-a4c684761e7a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:45.584762 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.584731    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr"
Jul 18 01:30:45.585246 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.584775    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8775cd-96bnr" event={"ID":"645ebe55-aec8-4145-bac5-a4c684761e7a","Type":"ContainerDied","Data":"87ee425e6f1b81c71f95e1e13e3a7195198f1b5575345b915fe49ea2b4a8092c"}
Jul 18 01:30:45.585246 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.584824    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87ee425e6f1b81c71f95e1e13e3a7195198f1b5575345b915fe49ea2b4a8092c"
Jul 18 01:30:45.734328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.734298    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:45.910343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.910240    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41408393-2234-427c-97bf-f5929d6a92ed-maas-api-service-ca\") pod \"41408393-2234-427c-97bf-f5929d6a92ed\" (UID: \"41408393-2234-427c-97bf-f5929d6a92ed\") "
Jul 18 01:30:45.910600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:45.910577    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41408393-2234-427c-97bf-f5929d6a92ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41408393-2234-427c-97bf-f5929d6a92ed" (UID: "41408393-2234-427c-97bf-f5929d6a92ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:46.011561 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.011527    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41408393-2234-427c-97bf-f5929d6a92ed-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:46.593542 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.593510    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"
Jul 18 01:30:46.594031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.593561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6" event={"ID":"41408393-2234-427c-97bf-f5929d6a92ed","Type":"ContainerDied","Data":"049a76c022a957c72d3447ead767a46514db3449873d04187f6d3ba1f1c18cf8"}
Jul 18 01:30:46.594031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.593636    2567 scope.go:117] "RemoveContainer" containerID="ae7482af30e9914e3f53d7c09a80024bf89da279f5894c1cc96b3b5ba8f6ec4d"
Jul 18 01:30:46.616524 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.616498    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"]
Jul 18 01:30:46.623338 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.623309    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-nj2g6"]
Jul 18 01:30:46.880073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.880049    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:46.920772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:46.920739    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41408393-2234-427c-97bf-f5929d6a92ed" path="/var/lib/kubelet/pods/41408393-2234-427c-97bf-f5929d6a92ed/volumes"
Jul 18 01:30:47.020329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020299    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-credential-socket\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020336    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-data\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020372    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-podinfo\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020398    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-socket\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020487    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-token\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020504    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-envoy\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020530    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jgpt\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-kube-api-access-6jgpt\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020556    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istiod-ca-cert\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020581    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-certs\") pod \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\" (UID: \"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74\") "
Jul 18 01:30:47.020875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020592    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:30:47.020875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.020862    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.021062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.021038    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:30:47.021123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.021070    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:47.021372 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.021125    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:30:47.021523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.021492    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-data" (OuterVolumeSpecName: "istio-data") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:30:47.022863 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.022836    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:30:47.022968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.022887    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:30:47.023359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.023332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-token" (OuterVolumeSpecName: "istio-token") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:30:47.023464 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.023332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-kube-api-access-6jgpt" (OuterVolumeSpecName: "kube-api-access-6jgpt") pod "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" (UID: "6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74"). InnerVolumeSpecName "kube-api-access-6jgpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121671    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121699    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121709    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jgpt\" (UniqueName: \"kubernetes.io/projected/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-kube-api-access-6jgpt\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121719    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121728    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121735    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121744    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.121755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.121752    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:47.600166 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.600129    2567 generic.go:356] "Generic (PLEG): container finished" podID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerID="d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b" exitCode=0
Jul 18 01:30:47.600666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.600183    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"
Jul 18 01:30:47.600666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.600214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" event={"ID":"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74","Type":"ContainerDied","Data":"d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b"}
Jul 18 01:30:47.600666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.600293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229" event={"ID":"6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74","Type":"ContainerDied","Data":"6902bdebcb5d0db64487c3b699ac90d339f7fa157491434fc153a15429f227a8"}
Jul 18 01:30:47.600666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.600327    2567 scope.go:117] "RemoveContainer" containerID="d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b"
Jul 18 01:30:47.610078 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.610061    2567 scope.go:117] "RemoveContainer" containerID="10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"
Jul 18 01:30:47.618669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.618652    2567 scope.go:117] "RemoveContainer" containerID="d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b"
Jul 18 01:30:47.619332 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:47.619026    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b\": container with ID starting with d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b not found: ID does not exist" containerID="d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b"
Jul 18 01:30:47.619332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.619069    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b"} err="failed to get container status \"d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b\": rpc error: code = NotFound desc = could not find container \"d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b\": container with ID starting with d08fadce5163b86668b4802d6c47b41b3d4d898da01df205e49582662aec000b not found: ID does not exist"
Jul 18 01:30:47.619332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.619093    2567 scope.go:117] "RemoveContainer" containerID="10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"
Jul 18 01:30:47.619548 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:47.619507    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d\": container with ID starting with 10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d not found: ID does not exist" containerID="10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"
Jul 18 01:30:47.619592 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.619542    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d"} err="failed to get container status \"10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d\": rpc error: code = NotFound desc = could not find container \"10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d\": container with ID starting with 10ead172ce57761e73cb0c86befb40c791f707602bd5564d255745a1dfcc043d not found: ID does not exist"
Jul 18 01:30:47.621766 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.621746    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"]
Jul 18 01:30:47.625632 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:47.625608    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-feded4-openshift-default-67978d9787-k7229"]
Jul 18 01:30:48.919216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:48.919173    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" path="/var/lib/kubelet/pods/6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74/volumes"
Jul 18 01:30:50.916095 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:50.916070    2567 scope.go:117] "RemoveContainer" containerID="269aaa44358f95daf11c6b44b1dd24e80261cc19631db43e2a863c8a30be0a37"
Jul 18 01:30:51.626026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:51.625990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" event={"ID":"b3067e99-a980-424d-a1f0-98b176a36655","Type":"ContainerStarted","Data":"411ac57f0deb708ea9a44439427af77035d373041273ef6938b6a7d8208531ae"}
Jul 18 01:30:51.643293 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:51.643225    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" podStartSLOduration=18.643210744 podStartE2EDuration="18.643210744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:30:51.640921484 +0000 UTC m=+3031.168426463" watchObservedRunningTime="2026-07-18 01:30:51.643210744 +0000 UTC m=+3031.170715704"
Jul 18 01:30:52.515203 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:52.515169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:30:52.635249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:52.635219    2567 generic.go:356] "Generic (PLEG): container finished" podID="b3067e99-a980-424d-a1f0-98b176a36655" containerID="411ac57f0deb708ea9a44439427af77035d373041273ef6938b6a7d8208531ae" exitCode=6
Jul 18 01:30:52.635423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:52.635301    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" event={"ID":"b3067e99-a980-424d-a1f0-98b176a36655","Type":"ContainerDied","Data":"411ac57f0deb708ea9a44439427af77035d373041273ef6938b6a7d8208531ae"}
Jul 18 01:30:52.635423 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:52.635377    2567 scope.go:117] "RemoveContainer" containerID="269aaa44358f95daf11c6b44b1dd24e80261cc19631db43e2a863c8a30be0a37"
Jul 18 01:30:52.635704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:52.635678    2567 scope.go:117] "RemoveContainer" containerID="411ac57f0deb708ea9a44439427af77035d373041273ef6938b6a7d8208531ae"
Jul 18 01:30:52.635955 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:30:52.635935    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-klwpk_odh-ai-gateway-infra(b3067e99-a980-424d-a1f0-98b176a36655)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" podUID="b3067e99-a980-424d-a1f0-98b176a36655"
Jul 18 01:30:52.651860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:52.651835    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"]
Jul 18 01:30:53.782883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:53.782857    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:53.874823 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:53.874789    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3067e99-a980-424d-a1f0-98b176a36655-maas-api-service-ca\") pod \"b3067e99-a980-424d-a1f0-98b176a36655\" (UID: \"b3067e99-a980-424d-a1f0-98b176a36655\") "
Jul 18 01:30:53.875144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:53.875121    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3067e99-a980-424d-a1f0-98b176a36655-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3067e99-a980-424d-a1f0-98b176a36655" (UID: "b3067e99-a980-424d-a1f0-98b176a36655"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:30:53.975848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:53.975760    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3067e99-a980-424d-a1f0-98b176a36655-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:30:54.649368 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:54.649339    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"
Jul 18 01:30:54.649530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:54.649387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk" event={"ID":"b3067e99-a980-424d-a1f0-98b176a36655","Type":"ContainerDied","Data":"bed4783aa8de34960de2c3c5e675863b6e14d15edcf020947e6f2abc05a252f6"}
Jul 18 01:30:54.649530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:54.649458    2567 scope.go:117] "RemoveContainer" containerID="411ac57f0deb708ea9a44439427af77035d373041273ef6938b6a7d8208531ae"
Jul 18 01:30:54.676163 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:54.676130    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"]
Jul 18 01:30:54.681909 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:54.681883    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-klwpk"]
Jul 18 01:30:54.920300 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:54.920201    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3067e99-a980-424d-a1f0-98b176a36655" path="/var/lib/kubelet/pods/b3067e99-a980-424d-a1f0-98b176a36655/volumes"
Jul 18 01:30:55.905121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:55.905081    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:30:57.094238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:30:57.094208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:01.406263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406223    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"]
Jul 18 01:31:01.406705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406681    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:01.406705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406699    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406712    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406721    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406732    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406742    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406754    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406759    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406774    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406779    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406785    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406790    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406797    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="645ebe55-aec8-4145-bac5-a4c684761e7a" containerName="revoke-keys"
Jul 18 01:31:01.406799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406801    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="645ebe55-aec8-4145-bac5-a4c684761e7a" containerName="revoke-keys"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406868    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406875    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406882    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406891    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="645ebe55-aec8-4145-bac5-a4c684761e7a" containerName="revoke-keys"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406901    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406907    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406913    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.407158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.406919    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3067e99-a980-424d-a1f0-98b176a36655" containerName="revoke-keys"
Jul 18 01:31:01.410005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.409986    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:01.412717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.412695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:31:01.412867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.412735    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:31:01.417090 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.417061    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"]
Jul 18 01:31:01.445282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.445226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36e490d8-97df-436b-b90b-dc074c05eb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42\" (UID: \"36e490d8-97df-436b-b90b-dc074c05eb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:01.546456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.546417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36e490d8-97df-436b-b90b-dc074c05eb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42\" (UID: \"36e490d8-97df-436b-b90b-dc074c05eb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:01.547109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.547090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36e490d8-97df-436b-b90b-dc074c05eb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42\" (UID: \"36e490d8-97df-436b-b90b-dc074c05eb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:01.721720 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.721632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:01.866783 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:01.866747    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"]
Jul 18 01:31:01.870177 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:01.870148    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e490d8_97df_436b_b90b_dc074c05eb81.slice/crio-3c945ca186bc6c5ee66cde8048edd78ed12bee96d7fb33be6d5f7ee0c9fec3c8 WatchSource:0}: Error finding container 3c945ca186bc6c5ee66cde8048edd78ed12bee96d7fb33be6d5f7ee0c9fec3c8: Status 404 returned error can't find the container with id 3c945ca186bc6c5ee66cde8048edd78ed12bee96d7fb33be6d5f7ee0c9fec3c8
Jul 18 01:31:02.695967 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:02.695931    2567 generic.go:356] "Generic (PLEG): container finished" podID="36e490d8-97df-436b-b90b-dc074c05eb81" containerID="d46c69322e48be071f35408fe113540a6cb371aee129769813965a691f23a28f" exitCode=6
Jul 18 01:31:02.696419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:02.696023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" event={"ID":"36e490d8-97df-436b-b90b-dc074c05eb81","Type":"ContainerDied","Data":"d46c69322e48be071f35408fe113540a6cb371aee129769813965a691f23a28f"}
Jul 18 01:31:02.696419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:02.696070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" event={"ID":"36e490d8-97df-436b-b90b-dc074c05eb81","Type":"ContainerStarted","Data":"3c945ca186bc6c5ee66cde8048edd78ed12bee96d7fb33be6d5f7ee0c9fec3c8"}
Jul 18 01:31:02.696419 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:02.696195    2567 scope.go:117] "RemoveContainer" containerID="d46c69322e48be071f35408fe113540a6cb371aee129769813965a691f23a28f"
Jul 18 01:31:02.897861 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:02.897827    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:03.021324 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:03.021292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:03.703385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:03.703294    2567 generic.go:356] "Generic (PLEG): container finished" podID="36e490d8-97df-436b-b90b-dc074c05eb81" containerID="45b9ba9d0317da3ba92c29267c16aba85ee8e96c35a9f3ed906757c2ad99737d" exitCode=6
Jul 18 01:31:03.703799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:03.703383    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" event={"ID":"36e490d8-97df-436b-b90b-dc074c05eb81","Type":"ContainerDied","Data":"45b9ba9d0317da3ba92c29267c16aba85ee8e96c35a9f3ed906757c2ad99737d"}
Jul 18 01:31:03.703799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:03.703443    2567 scope.go:117] "RemoveContainer" containerID="d46c69322e48be071f35408fe113540a6cb371aee129769813965a691f23a28f"
Jul 18 01:31:03.703799 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:03.703667    2567 scope.go:117] "RemoveContainer" containerID="45b9ba9d0317da3ba92c29267c16aba85ee8e96c35a9f3ed906757c2ad99737d"
Jul 18 01:31:03.703924 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:03.703892    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42_odh-ai-gateway-infra(36e490d8-97df-436b-b90b-dc074c05eb81)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" podUID="36e490d8-97df-436b-b90b-dc074c05eb81"
Jul 18 01:31:04.710821 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:04.710792    2567 scope.go:117] "RemoveContainer" containerID="45b9ba9d0317da3ba92c29267c16aba85ee8e96c35a9f3ed906757c2ad99737d"
Jul 18 01:31:04.711211 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:04.711023    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42_odh-ai-gateway-infra(36e490d8-97df-436b-b90b-dc074c05eb81)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" podUID="36e490d8-97df-436b-b90b-dc074c05eb81"
Jul 18 01:31:07.771240 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:07.771196    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"]
Jul 18 01:31:07.771651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:07.771468    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy" containerID="cri-o://3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2" gracePeriod=30
Jul 18 01:31:07.785669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:07.785637    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:08.010582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.010552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:08.330197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.330163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:08.610201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.610123    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"]
Jul 18 01:31:08.610530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.610516    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:08.610577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.610534    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:08.610577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.610558    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy"
Jul 18 01:31:08.610577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.610564    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fdb232a-1c9b-48ba-9b2a-a0dc5105dc74" containerName="istio-proxy"
Jul 18 01:31:08.610681 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.610635    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="41408393-2234-427c-97bf-f5929d6a92ed" containerName="revoke-keys"
Jul 18 01:31:08.615026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.615007    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.617485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.617466    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-6b3233-openshift-default-dockercfg-rnl2p\""
Jul 18 01:31:08.625543 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.625518    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"]
Jul 18 01:31:08.633743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.633713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:08.703988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.703957    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:08.713155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-certs\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-credential-socket\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713183    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-podinfo\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-envoy\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istiod-ca-cert\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxklq\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-kube-api-access-qxklq\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-data\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-socket\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.713584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.713538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-token\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.814462 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-certs\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.814462 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-credential-socket\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-podinfo\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814526    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-envoy\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814567    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istiod-ca-cert\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxklq\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-kube-api-access-qxklq\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-data\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814719    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-socket\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.814934    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-certs\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.815027    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-data\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815416 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.815082    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-token\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815416 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.815181    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-socket\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815416 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.815331    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istiod-ca-cert\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.815519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.815463    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-credential-socket\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.817352 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.817328    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-envoy\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.817741 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.817723    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-podinfo\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.823627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.823596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-token\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.824249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.824226    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxklq\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-kube-api-access-qxklq\") pod \"e2e-shared-a-6b3233-openshift-default-7946476559-h2d44\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") " pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:08.832120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.832092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:08.927215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:08.927124    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:09.071215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.071183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"]
Jul 18 01:31:09.073327 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:09.073251    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c31afe_b1b8_4c96_9598_0e44bc09a888.slice/crio-df122b91bddcb6d26454c4fed7ac449b6ccd553515d8db2f7d74c54af21000f4 WatchSource:0}: Error finding container df122b91bddcb6d26454c4fed7ac449b6ccd553515d8db2f7d74c54af21000f4: Status 404 returned error can't find the container with id df122b91bddcb6d26454c4fed7ac449b6ccd553515d8db2f7d74c54af21000f4
Jul 18 01:31:09.075656 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.075604    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:09.075752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.075732    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:09.075800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.075785    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:09.242395 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.242303    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"]
Jul 18 01:31:09.246302 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.246261    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:09.253584 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.253555    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"]
Jul 18 01:31:09.320773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.320724    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca4f8249-511d-455c-9ada-c636863e27c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9\" (UID: \"ca4f8249-511d-455c-9ada-c636863e27c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:09.422113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.422073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca4f8249-511d-455c-9ada-c636863e27c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9\" (UID: \"ca4f8249-511d-455c-9ada-c636863e27c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:09.422804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.422776    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca4f8249-511d-455c-9ada-c636863e27c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9\" (UID: \"ca4f8249-511d-455c-9ada-c636863e27c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:09.563259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.563221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:09.705635 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.705603    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"]
Jul 18 01:31:09.707184 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:09.707155    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4f8249_511d_455c_9ada_c636863e27c8.slice/crio-5e4e6c0f849e94ac897e03a810921ab1d2f4b7e44354e85da72dfe8ff0826d86 WatchSource:0}: Error finding container 5e4e6c0f849e94ac897e03a810921ab1d2f4b7e44354e85da72dfe8ff0826d86: Status 404 returned error can't find the container with id 5e4e6c0f849e94ac897e03a810921ab1d2f4b7e44354e85da72dfe8ff0826d86
Jul 18 01:31:09.740284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.740234    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" event={"ID":"ca4f8249-511d-455c-9ada-c636863e27c8","Type":"ContainerStarted","Data":"5e4e6c0f849e94ac897e03a810921ab1d2f4b7e44354e85da72dfe8ff0826d86"}
Jul 18 01:31:09.742015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.741987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" event={"ID":"d2c31afe-b1b8-4c96-9598-0e44bc09a888","Type":"ContainerStarted","Data":"1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"}
Jul 18 01:31:09.742179 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.742031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" event={"ID":"d2c31afe-b1b8-4c96-9598-0e44bc09a888","Type":"ContainerStarted","Data":"df122b91bddcb6d26454c4fed7ac449b6ccd553515d8db2f7d74c54af21000f4"}
Jul 18 01:31:09.761024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.760975    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" podStartSLOduration=1.760961099 podStartE2EDuration="1.760961099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:09.758665836 +0000 UTC m=+3049.286170838" watchObservedRunningTime="2026-07-18 01:31:09.760961099 +0000 UTC m=+3049.288466059"
Jul 18 01:31:09.927825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:09.927790    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:10.751001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:10.750963    2567 generic.go:356] "Generic (PLEG): container finished" podID="ca4f8249-511d-455c-9ada-c636863e27c8" containerID="7309c426c1552126474690c62deb94fd082cf761d14bc19853d130733aa55e16" exitCode=6
Jul 18 01:31:10.751189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:10.751057    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" event={"ID":"ca4f8249-511d-455c-9ada-c636863e27c8","Type":"ContainerDied","Data":"7309c426c1552126474690c62deb94fd082cf761d14bc19853d130733aa55e16"}
Jul 18 01:31:10.751314 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:10.751263    2567 scope.go:117] "RemoveContainer" containerID="7309c426c1552126474690c62deb94fd082cf761d14bc19853d130733aa55e16"
Jul 18 01:31:10.933448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:10.933418    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:11.642658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.642619    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:11.730122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.730038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:11.760113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.760073    2567 generic.go:356] "Generic (PLEG): container finished" podID="ca4f8249-511d-455c-9ada-c636863e27c8" containerID="7c96394ea2fe9ab77fd43b1d89673242c47851bfc529a69793604c16a2332630" exitCode=6
Jul 18 01:31:11.760331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.760199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" event={"ID":"ca4f8249-511d-455c-9ada-c636863e27c8","Type":"ContainerDied","Data":"7c96394ea2fe9ab77fd43b1d89673242c47851bfc529a69793604c16a2332630"}
Jul 18 01:31:11.760331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.760296    2567 scope.go:117] "RemoveContainer" containerID="7309c426c1552126474690c62deb94fd082cf761d14bc19853d130733aa55e16"
Jul 18 01:31:11.760331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.760316    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:11.760644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.760625    2567 scope.go:117] "RemoveContainer" containerID="7c96394ea2fe9ab77fd43b1d89673242c47851bfc529a69793604c16a2332630"
Jul 18 01:31:11.760888 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:11.760863    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9_odh-ai-gateway-infra(ca4f8249-511d-455c-9ada-c636863e27c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" podUID="ca4f8249-511d-455c-9ada-c636863e27c8"
Jul 18 01:31:11.762010 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.761976    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:11.814456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.814422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:11.910382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:11.910348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:12.318795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:12.318759    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:12.418851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:12.418815    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:12.768291 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:12.768174    2567 scope.go:117] "RemoveContainer" containerID="7c96394ea2fe9ab77fd43b1d89673242c47851bfc529a69793604c16a2332630"
Jul 18 01:31:12.768467 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:12.768449    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9_odh-ai-gateway-infra(ca4f8249-511d-455c-9ada-c636863e27c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" podUID="ca4f8249-511d-455c-9ada-c636863e27c8"
Jul 18 01:31:13.034817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.034796    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:31:13.161136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-credential-socket\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161178    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-data\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-podinfo\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161341    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc8zk\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-kube-api-access-gc8zk\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161369    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-envoy\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161426    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-socket\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161454    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-token\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161486    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc16d383-9251-42b3-bd41-0fbafe289aae-istiod-ca-cert\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.161652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.161524    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-certs\") pod \"bc16d383-9251-42b3-bd41-0fbafe289aae\" (UID: \"bc16d383-9251-42b3-bd41-0fbafe289aae\") "
Jul 18 01:31:13.162536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.162011    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:31:13.162536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.162054    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:31:13.162536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.162157    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:31:13.162536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.162488    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:31:13.162826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.162535    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc16d383-9251-42b3-bd41-0fbafe289aae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:31:13.163828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.163791    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:31:13.163940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.163834    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:31:13.163940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.163903    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:31:13.163940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.163911    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-kube-api-access-gc8zk" (OuterVolumeSpecName: "kube-api-access-gc8zk") pod "bc16d383-9251-42b3-bd41-0fbafe289aae" (UID: "bc16d383-9251-42b3-bd41-0fbafe289aae"). InnerVolumeSpecName "kube-api-access-gc8zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:31:13.262576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262539    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262572    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc8zk\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-kube-api-access-gc8zk\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262581    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262591    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262602    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262612    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc16d383-9251-42b3-bd41-0fbafe289aae-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262620    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262628    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.262838 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.262635    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc16d383-9251-42b3-bd41-0fbafe289aae-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:13.781607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.781577    2567 generic.go:356] "Generic (PLEG): container finished" podID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerID="3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2" exitCode=0
Jul 18 01:31:13.782012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.781668    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" event={"ID":"bc16d383-9251-42b3-bd41-0fbafe289aae","Type":"ContainerDied","Data":"3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2"}
Jul 18 01:31:13.782012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.781712    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"
Jul 18 01:31:13.782012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.781757    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp" event={"ID":"bc16d383-9251-42b3-bd41-0fbafe289aae","Type":"ContainerDied","Data":"68c138dd1f918dcb03abde1870461e8f068ba7fc55775aad6ea75f76db513e46"}
Jul 18 01:31:13.782012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.781787    2567 scope.go:117] "RemoveContainer" containerID="3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2"
Jul 18 01:31:13.792138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.792118    2567 scope.go:117] "RemoveContainer" containerID="9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"
Jul 18 01:31:13.800995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.800976    2567 scope.go:117] "RemoveContainer" containerID="3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2"
Jul 18 01:31:13.801253 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:13.801234    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2\": container with ID starting with 3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2 not found: ID does not exist" containerID="3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2"
Jul 18 01:31:13.801340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.801260    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2"} err="failed to get container status \"3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2\": rpc error: code = NotFound desc = could not find container \"3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2\": container with ID starting with 3d4ae6d6bd59a731df0ac836db1c4c19544aadd049af9e16d2a16a9b557f16c2 not found: ID does not exist"
Jul 18 01:31:13.801340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.801301    2567 scope.go:117] "RemoveContainer" containerID="9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"
Jul 18 01:31:13.801545 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:13.801530    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b\": container with ID starting with 9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b not found: ID does not exist" containerID="9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"
Jul 18 01:31:13.801587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.801549    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b"} err="failed to get container status \"9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b\": rpc error: code = NotFound desc = could not find container \"9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b\": container with ID starting with 9c617f2b608490e1facdd216657df89a5e927f0a394f19070e097739fe557e6b not found: ID does not exist"
Jul 18 01:31:13.805226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.805204    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"]
Jul 18 01:31:13.810001 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:13.809980    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8775cd-openshift-default-86bbd857c7-9lknp"]
Jul 18 01:31:14.920170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:14.920138    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" path="/var/lib/kubelet/pods/bc16d383-9251-42b3-bd41-0fbafe289aae/volumes"
Jul 18 01:31:16.117019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.116963    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:16.434473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.434384    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:16.607138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.607100    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:31:16.711653 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.711558    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"]
Jul 18 01:31:16.712103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.712081    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy"
Jul 18 01:31:16.712219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.712108    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy"
Jul 18 01:31:16.712219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.712118    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy"
Jul 18 01:31:16.712219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.712126    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy"
Jul 18 01:31:16.712403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.712246    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy"
Jul 18 01:31:16.712403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.712260    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc16d383-9251-42b3-bd41-0fbafe289aae" containerName="istio-proxy"
Jul 18 01:31:16.716964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.716939    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.719716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.719695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b17102-openshift-default-dockercfg-tp6vc\""
Jul 18 01:31:16.728000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.727963    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"]
Jul 18 01:31:16.736318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.736263    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:16.796189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f690faca-2ef9-406b-a34d-96e24d9dd24d-istiod-ca-cert\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-data\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796237    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-envoy\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-socket\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-token\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796453 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796409    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpbmb\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-kube-api-access-hpbmb\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-credential-socket\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-certs\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.796680 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.796566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-podinfo\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-data\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-envoy\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-socket\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897403    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-token\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897455    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpbmb\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-kube-api-access-hpbmb\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-credential-socket\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-certs\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-podinfo\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f690faca-2ef9-406b-a34d-96e24d9dd24d-istiod-ca-cert\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-data\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-credential-socket\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897918    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-certs\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.897959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.897951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-socket\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.898227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.898208    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f690faca-2ef9-406b-a34d-96e24d9dd24d-istiod-ca-cert\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.900039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.900015    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-envoy\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.900366 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.900349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-podinfo\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.905640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.905612    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:16.905755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.905734    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-token\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.905858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.905837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpbmb\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-kube-api-access-hpbmb\") pod \"e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") " pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:16.914048 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:16.914023    2567 scope.go:117] "RemoveContainer" containerID="45b9ba9d0317da3ba92c29267c16aba85ee8e96c35a9f3ed906757c2ad99737d"
Jul 18 01:31:17.030139 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.030050    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:17.179259 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.179236    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"]
Jul 18 01:31:17.180677 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:17.180648    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf690faca_2ef9_406b_a34d_96e24d9dd24d.slice/crio-5b2064506cd71ab566f9b043817800fe79200e11d73d3d7e8cb7a6d1d3caa4ed WatchSource:0}: Error finding container 5b2064506cd71ab566f9b043817800fe79200e11d73d3d7e8cb7a6d1d3caa4ed: Status 404 returned error can't find the container with id 5b2064506cd71ab566f9b043817800fe79200e11d73d3d7e8cb7a6d1d3caa4ed
Jul 18 01:31:17.182832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.182802    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:17.182909 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.182865    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:17.182909 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.182897    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:17.807207 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.807169    2567 generic.go:356] "Generic (PLEG): container finished" podID="36e490d8-97df-436b-b90b-dc074c05eb81" containerID="9433d2c9607013028ee83662e01893189de5bb9f418deb2b94b60cd92d0fedb2" exitCode=6
Jul 18 01:31:17.807432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.807244    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" event={"ID":"36e490d8-97df-436b-b90b-dc074c05eb81","Type":"ContainerDied","Data":"9433d2c9607013028ee83662e01893189de5bb9f418deb2b94b60cd92d0fedb2"}
Jul 18 01:31:17.807432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.807383    2567 scope.go:117] "RemoveContainer" containerID="45b9ba9d0317da3ba92c29267c16aba85ee8e96c35a9f3ed906757c2ad99737d"
Jul 18 01:31:17.807695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.807678    2567 scope.go:117] "RemoveContainer" containerID="9433d2c9607013028ee83662e01893189de5bb9f418deb2b94b60cd92d0fedb2"
Jul 18 01:31:17.807945 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:17.807923    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42_odh-ai-gateway-infra(36e490d8-97df-436b-b90b-dc074c05eb81)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" podUID="36e490d8-97df-436b-b90b-dc074c05eb81"
Jul 18 01:31:17.809096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.809070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" event={"ID":"f690faca-2ef9-406b-a34d-96e24d9dd24d","Type":"ContainerStarted","Data":"938996b8e2217a38ff44e72ec5e7c7e9bf932c5574d88cef7009eaf624efe5c2"}
Jul 18 01:31:17.809199 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.809115    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" event={"ID":"f690faca-2ef9-406b-a34d-96e24d9dd24d","Type":"ContainerStarted","Data":"5b2064506cd71ab566f9b043817800fe79200e11d73d3d7e8cb7a6d1d3caa4ed"}
Jul 18 01:31:17.845285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:17.845209    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podStartSLOduration=1.845191942 podStartE2EDuration="1.845191942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:17.843387905 +0000 UTC m=+3057.370892864" watchObservedRunningTime="2026-07-18 01:31:17.845191942 +0000 UTC m=+3057.372696902"
Jul 18 01:31:18.030691 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:18.030605    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:18.035689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:18.035663    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:18.816411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:18.816381    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:18.817478 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:18.817462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:18.837860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:18.837828    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"]
Jul 18 01:31:19.038694 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.038669    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:19.120250 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.120168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36e490d8-97df-436b-b90b-dc074c05eb81-maas-api-service-ca\") pod \"36e490d8-97df-436b-b90b-dc074c05eb81\" (UID: \"36e490d8-97df-436b-b90b-dc074c05eb81\") "
Jul 18 01:31:19.120577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.120556    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e490d8-97df-436b-b90b-dc074c05eb81-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36e490d8-97df-436b-b90b-dc074c05eb81" (UID: "36e490d8-97df-436b-b90b-dc074c05eb81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:31:19.221013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.220979    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36e490d8-97df-436b-b90b-dc074c05eb81-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:19.729953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.729920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:19.798300 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.798250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:19.826449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.826421    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"
Jul 18 01:31:19.826925 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.826460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42" event={"ID":"36e490d8-97df-436b-b90b-dc074c05eb81","Type":"ContainerDied","Data":"3c945ca186bc6c5ee66cde8048edd78ed12bee96d7fb33be6d5f7ee0c9fec3c8"}
Jul 18 01:31:19.826925 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.826524    2567 scope.go:117] "RemoveContainer" containerID="9433d2c9607013028ee83662e01893189de5bb9f418deb2b94b60cd92d0fedb2"
Jul 18 01:31:19.865294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.862892    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"]
Jul 18 01:31:19.879571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.879537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:19.883044 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:19.883007    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jdf42"]
Jul 18 01:31:20.354709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:20.354650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:20.454690 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:20.454654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:20.558215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:20.558174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:20.920161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:20.920128    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" path="/var/lib/kubelet/pods/36e490d8-97df-436b-b90b-dc074c05eb81/volumes"
Jul 18 01:31:24.213637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.213597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:24.864088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.862361    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerID="1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f" exitCode=0
Jul 18 01:31:24.864088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.862460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" event={"ID":"d2c31afe-b1b8-4c96-9598-0e44bc09a888","Type":"ContainerDied","Data":"1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"}
Jul 18 01:31:24.864088 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.862921    2567 scope.go:117] "RemoveContainer" containerID="1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"
Jul 18 01:31:24.865457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.865321    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:24.865457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.865439    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:24.865657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.865495    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:24.913977 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:24.913947    2567 scope.go:117] "RemoveContainer" containerID="7c96394ea2fe9ab77fd43b1d89673242c47851bfc529a69793604c16a2332630"
Jul 18 01:31:25.884426 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:25.881935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" event={"ID":"ca4f8249-511d-455c-9ada-c636863e27c8","Type":"ContainerStarted","Data":"d2bce3c3d2aa0f1e46b3e930346adc22c27bc874dd23478b4bff4cde7a1f8eee"}
Jul 18 01:31:25.886397 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:25.884865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" event={"ID":"d2c31afe-b1b8-4c96-9598-0e44bc09a888","Type":"ContainerStarted","Data":"362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc"}
Jul 18 01:31:25.926792 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:25.926742    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" podStartSLOduration=16.926725282 podStartE2EDuration="16.926725282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:25.905663806 +0000 UTC m=+3065.433168767" watchObservedRunningTime="2026-07-18 01:31:25.926725282 +0000 UTC m=+3065.454230242"
Jul 18 01:31:25.927611 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:25.927586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:25.928835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:25.928609    2567 patch_prober.go:28] interesting pod/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Jul 18 01:31:25.928835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:25.928652    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Jul 18 01:31:26.463896 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.463851    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"]
Jul 18 01:31:26.464764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.464736    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:26.464903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.464893    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:26.465031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.465021    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:26.465106 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.465098    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:26.465313 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.465300    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:26.465411 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.465402    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:26.469357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.469323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:26.477906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.477863    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"]
Jul 18 01:31:26.601808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.601758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d34d6320-e040-4ef6-bd55-f06cc7ea3d46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7\" (UID: \"d34d6320-e040-4ef6-bd55-f06cc7ea3d46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:26.703205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.703148    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d34d6320-e040-4ef6-bd55-f06cc7ea3d46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7\" (UID: \"d34d6320-e040-4ef6-bd55-f06cc7ea3d46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:26.704096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.704064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d34d6320-e040-4ef6-bd55-f06cc7ea3d46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7\" (UID: \"d34d6320-e040-4ef6-bd55-f06cc7ea3d46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:26.760666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.760515    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:26.787467 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.787136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:26.928072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.928022    2567 patch_prober.go:28] interesting pod/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Jul 18 01:31:26.928522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.928094    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Jul 18 01:31:26.928522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.928126    2567 generic.go:356] "Generic (PLEG): container finished" podID="ca4f8249-511d-455c-9ada-c636863e27c8" containerID="d2bce3c3d2aa0f1e46b3e930346adc22c27bc874dd23478b4bff4cde7a1f8eee" exitCode=6
Jul 18 01:31:26.931247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.931218    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" event={"ID":"ca4f8249-511d-455c-9ada-c636863e27c8","Type":"ContainerDied","Data":"d2bce3c3d2aa0f1e46b3e930346adc22c27bc874dd23478b4bff4cde7a1f8eee"}
Jul 18 01:31:26.931484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.931299    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"]
Jul 18 01:31:26.931484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:26.931339    2567 scope.go:117] "RemoveContainer" containerID="7c96394ea2fe9ab77fd43b1d89673242c47851bfc529a69793604c16a2332630"
Jul 18 01:31:27.045515 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.045476    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"]
Jul 18 01:31:27.046145 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:27.046118    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34d6320_e040_4ef6_bd55_f06cc7ea3d46.slice/crio-5ad5bb330b66e6fdc9126f712fc40139f8b12826d96df872f4a6bcc94cdd363a WatchSource:0}: Error finding container 5ad5bb330b66e6fdc9126f712fc40139f8b12826d96df872f4a6bcc94cdd363a: Status 404 returned error can't find the container with id 5ad5bb330b66e6fdc9126f712fc40139f8b12826d96df872f4a6bcc94cdd363a
Jul 18 01:31:27.221952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.221926    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:27.310297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.308852    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca4f8249-511d-455c-9ada-c636863e27c8-maas-api-service-ca\") pod \"ca4f8249-511d-455c-9ada-c636863e27c8\" (UID: \"ca4f8249-511d-455c-9ada-c636863e27c8\") "
Jul 18 01:31:27.310297 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.309749    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca4f8249-511d-455c-9ada-c636863e27c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca4f8249-511d-455c-9ada-c636863e27c8" (UID: "ca4f8249-511d-455c-9ada-c636863e27c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:31:27.409940 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.409907    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca4f8249-511d-455c-9ada-c636863e27c8-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:27.927943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.927904    2567 patch_prober.go:28] interesting pod/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Jul 18 01:31:27.928140 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.927965    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Jul 18 01:31:27.935111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.935086    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"
Jul 18 01:31:27.935261 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.935126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9" event={"ID":"ca4f8249-511d-455c-9ada-c636863e27c8","Type":"ContainerDied","Data":"5e4e6c0f849e94ac897e03a810921ab1d2f4b7e44354e85da72dfe8ff0826d86"}
Jul 18 01:31:27.935261 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.935196    2567 scope.go:117] "RemoveContainer" containerID="d2bce3c3d2aa0f1e46b3e930346adc22c27bc874dd23478b4bff4cde7a1f8eee"
Jul 18 01:31:27.936992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.936969    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" event={"ID":"d34d6320-e040-4ef6-bd55-f06cc7ea3d46","Type":"ContainerStarted","Data":"bb33e4c9719648e268f7c87e59460fa0754910b94626d058e2172d4389a160ad"}
Jul 18 01:31:27.937109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.937025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" event={"ID":"d34d6320-e040-4ef6-bd55-f06cc7ea3d46","Type":"ContainerStarted","Data":"5ad5bb330b66e6fdc9126f712fc40139f8b12826d96df872f4a6bcc94cdd363a"}
Jul 18 01:31:27.953256 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.953194    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" podStartSLOduration=1.953172256 podStartE2EDuration="1.953172256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:27.95106983 +0000 UTC m=+3067.478574791" watchObservedRunningTime="2026-07-18 01:31:27.953172256 +0000 UTC m=+3067.480677218"
Jul 18 01:31:27.969360 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.969322    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"]
Jul 18 01:31:27.979927 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:27.979884    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-dppr9"]
Jul 18 01:31:28.920097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:28.920052    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" path="/var/lib/kubelet/pods/ca4f8249-511d-455c-9ada-c636863e27c8/volumes"
Jul 18 01:31:28.927803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:28.927767    2567 patch_prober.go:28] interesting pod/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Jul 18 01:31:28.927968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:28.927823    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Jul 18 01:31:28.944228 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:28.944195    2567 generic.go:356] "Generic (PLEG): container finished" podID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerID="bb33e4c9719648e268f7c87e59460fa0754910b94626d058e2172d4389a160ad" exitCode=6
Jul 18 01:31:28.944677 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:28.944302    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" event={"ID":"d34d6320-e040-4ef6-bd55-f06cc7ea3d46","Type":"ContainerDied","Data":"bb33e4c9719648e268f7c87e59460fa0754910b94626d058e2172d4389a160ad"}
Jul 18 01:31:28.944677 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:28.944527    2567 scope.go:117] "RemoveContainer" containerID="bb33e4c9719648e268f7c87e59460fa0754910b94626d058e2172d4389a160ad"
Jul 18 01:31:29.932543 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:29.932514    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:29.933987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:29.933959    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:31:29.959381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:29.959341    2567 generic.go:356] "Generic (PLEG): container finished" podID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerID="95d2b56f1c366b2aac6b7d728c1bfff987d41f198f4e623e3b0f653cda00c6ae" exitCode=6
Jul 18 01:31:29.959848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:29.959431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" event={"ID":"d34d6320-e040-4ef6-bd55-f06cc7ea3d46","Type":"ContainerDied","Data":"95d2b56f1c366b2aac6b7d728c1bfff987d41f198f4e623e3b0f653cda00c6ae"}
Jul 18 01:31:29.959848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:29.959497    2567 scope.go:117] "RemoveContainer" containerID="bb33e4c9719648e268f7c87e59460fa0754910b94626d058e2172d4389a160ad"
Jul 18 01:31:29.959848 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:29.959652    2567 scope.go:117] "RemoveContainer" containerID="95d2b56f1c366b2aac6b7d728c1bfff987d41f198f4e623e3b0f653cda00c6ae"
Jul 18 01:31:29.959984 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:29.959920    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7_odh-ai-gateway-infra(d34d6320-e040-4ef6-bd55-f06cc7ea3d46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46"
Jul 18 01:31:30.974991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:30.974962    2567 scope.go:117] "RemoveContainer" containerID="95d2b56f1c366b2aac6b7d728c1bfff987d41f198f4e623e3b0f653cda00c6ae"
Jul 18 01:31:30.975493 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:30.975213    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7_odh-ai-gateway-infra(d34d6320-e040-4ef6-bd55-f06cc7ea3d46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46"
Jul 18 01:31:32.812064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.812029    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:31:32.986436 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.986397    2567 generic.go:356] "Generic (PLEG): container finished" podID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerID="938996b8e2217a38ff44e72ec5e7c7e9bf932c5574d88cef7009eaf624efe5c2" exitCode=0
Jul 18 01:31:32.986611 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.986468    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" event={"ID":"f690faca-2ef9-406b-a34d-96e24d9dd24d","Type":"ContainerDied","Data":"938996b8e2217a38ff44e72ec5e7c7e9bf932c5574d88cef7009eaf624efe5c2"}
Jul 18 01:31:32.986893 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.986879    2567 scope.go:117] "RemoveContainer" containerID="938996b8e2217a38ff44e72ec5e7c7e9bf932c5574d88cef7009eaf624efe5c2"
Jul 18 01:31:32.987937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.987906    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:32.988000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.987983    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:32.988042 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:32.988032    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 18 01:31:33.000660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.000623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:33.246809 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.246779    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:33.349426 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.349327    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:33.817313 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.817262    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:33.862237 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862205    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"]
Jul 18 01:31:33.862651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862637    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:33.862697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862652    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:33.862697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862666    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:33.862697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862673    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:33.862697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862679    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:33.862697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862684    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:33.862904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862777    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:33.862904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862789    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e490d8-97df-436b-b90b-dc074c05eb81" containerName="revoke-keys"
Jul 18 01:31:33.862904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.862798    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:33.866369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.866345    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:33.873160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.872951    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"]
Jul 18 01:31:33.977709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.977659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8950684-09de-4b8d-a40f-9e5e21227152-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8\" (UID: \"b8950684-09de-4b8d-a40f-9e5e21227152\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:33.994020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:33.993984    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" event={"ID":"f690faca-2ef9-406b-a34d-96e24d9dd24d","Type":"ContainerStarted","Data":"ceca33f8824787962d0551c1d26311b99b34591fc34269b1b1cb053143c01f59"}
Jul 18 01:31:34.031178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.031144    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:34.032529 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.032500    2567 patch_prober.go:28] interesting pod/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Jul 18 01:31:34.032658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.032553    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Jul 18 01:31:34.079058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.078945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8950684-09de-4b8d-a40f-9e5e21227152-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8\" (UID: \"b8950684-09de-4b8d-a40f-9e5e21227152\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:34.079956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.079930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8950684-09de-4b8d-a40f-9e5e21227152-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8\" (UID: \"b8950684-09de-4b8d-a40f-9e5e21227152\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:34.180195 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.180142    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:34.346441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:34.346400    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"]
Jul 18 01:31:34.347364 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:34.347323    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8950684_09de_4b8d_a40f_9e5e21227152.slice/crio-7ac4ec48076907a3f6fda83cd5a22ecb76c972e6d23000df37de0848fb521c7f WatchSource:0}: Error finding container 7ac4ec48076907a3f6fda83cd5a22ecb76c972e6d23000df37de0848fb521c7f: Status 404 returned error can't find the container with id 7ac4ec48076907a3f6fda83cd5a22ecb76c972e6d23000df37de0848fb521c7f
Jul 18 01:31:35.001489 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:35.001316    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" event={"ID":"b8950684-09de-4b8d-a40f-9e5e21227152","Type":"ContainerStarted","Data":"d8789b93a1fd65006652dfdcdb8745f43a6631decd2d79806feeacfef5f75c5e"}
Jul 18 01:31:35.001489 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:35.001380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" event={"ID":"b8950684-09de-4b8d-a40f-9e5e21227152","Type":"ContainerStarted","Data":"7ac4ec48076907a3f6fda83cd5a22ecb76c972e6d23000df37de0848fb521c7f"}
Jul 18 01:31:35.020456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:35.020398    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" podStartSLOduration=2.020383149 podStartE2EDuration="2.020383149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:35.018175728 +0000 UTC m=+3074.545680689" watchObservedRunningTime="2026-07-18 01:31:35.020383149 +0000 UTC m=+3074.547888108"
Jul 18 01:31:35.030974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:35.030944    2567 patch_prober.go:28] interesting pod/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Jul 18 01:31:35.031127 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:35.031003    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Jul 18 01:31:35.791620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:35.791584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:36.008479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:36.008445    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8950684-09de-4b8d-a40f-9e5e21227152" containerID="d8789b93a1fd65006652dfdcdb8745f43a6631decd2d79806feeacfef5f75c5e" exitCode=6
Jul 18 01:31:36.008978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:36.008527    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" event={"ID":"b8950684-09de-4b8d-a40f-9e5e21227152","Type":"ContainerDied","Data":"d8789b93a1fd65006652dfdcdb8745f43a6631decd2d79806feeacfef5f75c5e"}
Jul 18 01:31:36.008978 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:36.008766    2567 scope.go:117] "RemoveContainer" containerID="d8789b93a1fd65006652dfdcdb8745f43a6631decd2d79806feeacfef5f75c5e"
Jul 18 01:31:36.030808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:36.030774    2567 patch_prober.go:28] interesting pod/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Jul 18 01:31:36.030947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:36.030834    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Jul 18 01:31:37.015885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:37.015853    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8950684-09de-4b8d-a40f-9e5e21227152" containerID="b63efaf2333d156582d290878d594b03e17e37f80205585e7049d8a092c258f4" exitCode=6
Jul 18 01:31:37.016347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:37.015948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" event={"ID":"b8950684-09de-4b8d-a40f-9e5e21227152","Type":"ContainerDied","Data":"b63efaf2333d156582d290878d594b03e17e37f80205585e7049d8a092c258f4"}
Jul 18 01:31:37.016347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:37.016007    2567 scope.go:117] "RemoveContainer" containerID="d8789b93a1fd65006652dfdcdb8745f43a6631decd2d79806feeacfef5f75c5e"
Jul 18 01:31:37.016347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:37.016142    2567 scope.go:117] "RemoveContainer" containerID="b63efaf2333d156582d290878d594b03e17e37f80205585e7049d8a092c258f4"
Jul 18 01:31:37.016489 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:37.016414    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8_odh-ai-gateway-infra(b8950684-09de-4b8d-a40f-9e5e21227152)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" podUID="b8950684-09de-4b8d-a40f-9e5e21227152"
Jul 18 01:31:37.031000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:37.030970    2567 patch_prober.go:28] interesting pod/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Jul 18 01:31:37.031150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:37.031020    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Jul 18 01:31:38.023326 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:38.023297    2567 scope.go:117] "RemoveContainer" containerID="b63efaf2333d156582d290878d594b03e17e37f80205585e7049d8a092c258f4"
Jul 18 01:31:38.023712 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:38.023502    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8_odh-ai-gateway-infra(b8950684-09de-4b8d-a40f-9e5e21227152)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" podUID="b8950684-09de-4b8d-a40f-9e5e21227152"
Jul 18 01:31:38.030619 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:38.030588    2567 patch_prober.go:28] interesting pod/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Jul 18 01:31:38.030770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:38.030633    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Jul 18 01:31:39.035197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:39.035168    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:39.035588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:39.035516    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:39.036214 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:39.036194    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:31:42.224175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:42.224138    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:42.323795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:42.323761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:42.913900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:42.913863    2567 scope.go:117] "RemoveContainer" containerID="95d2b56f1c366b2aac6b7d728c1bfff987d41f198f4e623e3b0f653cda00c6ae"
Jul 18 01:31:44.058027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:44.057995    2567 generic.go:356] "Generic (PLEG): container finished" podID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerID="7c3f48046fc36b3a4cc900e796d94c05e79bf4bc9d350c239d89e40b64f76d5c" exitCode=6
Jul 18 01:31:44.058448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:44.058065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" event={"ID":"d34d6320-e040-4ef6-bd55-f06cc7ea3d46","Type":"ContainerDied","Data":"7c3f48046fc36b3a4cc900e796d94c05e79bf4bc9d350c239d89e40b64f76d5c"}
Jul 18 01:31:44.058448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:44.058128    2567 scope.go:117] "RemoveContainer" containerID="95d2b56f1c366b2aac6b7d728c1bfff987d41f198f4e623e3b0f653cda00c6ae"
Jul 18 01:31:44.058554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:44.058508    2567 scope.go:117] "RemoveContainer" containerID="7c3f48046fc36b3a4cc900e796d94c05e79bf4bc9d350c239d89e40b64f76d5c"
Jul 18 01:31:44.058778 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:44.058754    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7_odh-ai-gateway-infra(d34d6320-e040-4ef6-bd55-f06cc7ea3d46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46"
Jul 18 01:31:45.081454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:45.081418    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"]
Jul 18 01:31:45.223992 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:45.223963    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:45.280476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:45.280444    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d34d6320-e040-4ef6-bd55-f06cc7ea3d46-maas-api-service-ca\") pod \"d34d6320-e040-4ef6-bd55-f06cc7ea3d46\" (UID: \"d34d6320-e040-4ef6-bd55-f06cc7ea3d46\") "
Jul 18 01:31:45.280853 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:45.280833    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d34d6320-e040-4ef6-bd55-f06cc7ea3d46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d34d6320-e040-4ef6-bd55-f06cc7ea3d46" (UID: "d34d6320-e040-4ef6-bd55-f06cc7ea3d46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:31:45.381948 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:45.381848    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d34d6320-e040-4ef6-bd55-f06cc7ea3d46-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:46.072661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:46.072629    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"
Jul 18 01:31:46.072842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:46.072669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7" event={"ID":"d34d6320-e040-4ef6-bd55-f06cc7ea3d46","Type":"ContainerDied","Data":"5ad5bb330b66e6fdc9126f712fc40139f8b12826d96df872f4a6bcc94cdd363a"}
Jul 18 01:31:46.072842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:46.072728    2567 scope.go:117] "RemoveContainer" containerID="7c3f48046fc36b3a4cc900e796d94c05e79bf4bc9d350c239d89e40b64f76d5c"
Jul 18 01:31:46.098663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:46.098632    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"]
Jul 18 01:31:46.102645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:46.102621    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9r4z7"]
Jul 18 01:31:46.919106 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:46.919076    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" path="/var/lib/kubelet/pods/d34d6320-e040-4ef6-bd55-f06cc7ea3d46/volumes"
Jul 18 01:31:51.443699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.443663    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"]
Jul 18 01:31:51.444087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444047    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444058    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444066    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:51.444087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444072    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:51.444087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444078    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444083    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444097    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444102    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444166    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444172    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:51.444310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.444182    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca4f8249-511d-455c-9ada-c636863e27c8" containerName="revoke-keys"
Jul 18 01:31:51.448522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.448505    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:31:51.455262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.455237    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"]
Jul 18 01:31:51.533897 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.533864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1e6928b-71a3-4944-9d62-88402e6ff054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8\" (UID: \"a1e6928b-71a3-4944-9d62-88402e6ff054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:31:51.634382 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.634346    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1e6928b-71a3-4944-9d62-88402e6ff054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8\" (UID: \"a1e6928b-71a3-4944-9d62-88402e6ff054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:31:51.635028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.635005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1e6928b-71a3-4944-9d62-88402e6ff054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8\" (UID: \"a1e6928b-71a3-4944-9d62-88402e6ff054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:31:51.724748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.724672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:31:51.761156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.761114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:31:51.923439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:51.923387    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"]
Jul 18 01:31:51.925606 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:51.925576    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e6928b_71a3_4944_9d62_88402e6ff054.slice/crio-f6140ca6769a4794b3ad11df2b3be7e11ca13e43155acaea7e2c33df61dc239c WatchSource:0}: Error finding container f6140ca6769a4794b3ad11df2b3be7e11ca13e43155acaea7e2c33df61dc239c: Status 404 returned error can't find the container with id f6140ca6769a4794b3ad11df2b3be7e11ca13e43155acaea7e2c33df61dc239c
Jul 18 01:31:52.108883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:52.108849    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" event={"ID":"a1e6928b-71a3-4944-9d62-88402e6ff054","Type":"ContainerStarted","Data":"f6140ca6769a4794b3ad11df2b3be7e11ca13e43155acaea7e2c33df61dc239c"}
Jul 18 01:31:52.913456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:52.913370    2567 scope.go:117] "RemoveContainer" containerID="b63efaf2333d156582d290878d594b03e17e37f80205585e7049d8a092c258f4"
Jul 18 01:31:53.117402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:53.117366    2567 generic.go:356] "Generic (PLEG): container finished" podID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerID="61de890ab718f7deed22a17cd21f9cc28c3d9351d6bc1a2107eb04e96cede010" exitCode=6
Jul 18 01:31:53.117602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:53.117419    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" event={"ID":"a1e6928b-71a3-4944-9d62-88402e6ff054","Type":"ContainerDied","Data":"61de890ab718f7deed22a17cd21f9cc28c3d9351d6bc1a2107eb04e96cede010"}
Jul 18 01:31:53.117677 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:53.117664    2567 scope.go:117] "RemoveContainer" containerID="61de890ab718f7deed22a17cd21f9cc28c3d9351d6bc1a2107eb04e96cede010"
Jul 18 01:31:54.124658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.124567    2567 generic.go:356] "Generic (PLEG): container finished" podID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerID="388459fec11773aa3cf170ec0cc9084b6b525b3946fe3f747630bec290a06713" exitCode=6
Jul 18 01:31:54.124658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.124645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" event={"ID":"a1e6928b-71a3-4944-9d62-88402e6ff054","Type":"ContainerDied","Data":"388459fec11773aa3cf170ec0cc9084b6b525b3946fe3f747630bec290a06713"}
Jul 18 01:31:54.125145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.124702    2567 scope.go:117] "RemoveContainer" containerID="61de890ab718f7deed22a17cd21f9cc28c3d9351d6bc1a2107eb04e96cede010"
Jul 18 01:31:54.125145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.124913    2567 scope.go:117] "RemoveContainer" containerID="388459fec11773aa3cf170ec0cc9084b6b525b3946fe3f747630bec290a06713"
Jul 18 01:31:54.125259 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:54.125143    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8_odh-ai-gateway-infra(a1e6928b-71a3-4944-9d62-88402e6ff054)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054"
Jul 18 01:31:54.126679 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.126659    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8950684-09de-4b8d-a40f-9e5e21227152" containerID="b16a1a0cde99bd069e2e54dc09848c88263f4a0a80ba8bd58cee516f00e516b1" exitCode=6
Jul 18 01:31:54.126811 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.126735    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" event={"ID":"b8950684-09de-4b8d-a40f-9e5e21227152","Type":"ContainerDied","Data":"b16a1a0cde99bd069e2e54dc09848c88263f4a0a80ba8bd58cee516f00e516b1"}
Jul 18 01:31:54.127058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.127042    2567 scope.go:117] "RemoveContainer" containerID="b16a1a0cde99bd069e2e54dc09848c88263f4a0a80ba8bd58cee516f00e516b1"
Jul 18 01:31:54.127263 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:54.127245    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8_odh-ai-gateway-infra(b8950684-09de-4b8d-a40f-9e5e21227152)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" podUID="b8950684-09de-4b8d-a40f-9e5e21227152"
Jul 18 01:31:54.136484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:54.136469    2567 scope.go:117] "RemoveContainer" containerID="b63efaf2333d156582d290878d594b03e17e37f80205585e7049d8a092c258f4"
Jul 18 01:31:55.105329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.105287    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:31:55.136327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.136300    2567 scope.go:117] "RemoveContainer" containerID="388459fec11773aa3cf170ec0cc9084b6b525b3946fe3f747630bec290a06713"
Jul 18 01:31:55.136726 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:31:55.136571    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8_odh-ai-gateway-infra(a1e6928b-71a3-4944-9d62-88402e6ff054)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054"
Jul 18 01:31:55.163101 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.163061    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"]
Jul 18 01:31:55.314763 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.314730    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:55.377706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.377612    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8950684-09de-4b8d-a40f-9e5e21227152-maas-api-service-ca\") pod \"b8950684-09de-4b8d-a40f-9e5e21227152\" (UID: \"b8950684-09de-4b8d-a40f-9e5e21227152\") "
Jul 18 01:31:55.378002 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.377965    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8950684-09de-4b8d-a40f-9e5e21227152-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b8950684-09de-4b8d-a40f-9e5e21227152" (UID: "b8950684-09de-4b8d-a40f-9e5e21227152"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:31:55.479061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:55.479024    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8950684-09de-4b8d-a40f-9e5e21227152-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:31:56.143353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:56.143320    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"
Jul 18 01:31:56.143353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:56.143332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8" event={"ID":"b8950684-09de-4b8d-a40f-9e5e21227152","Type":"ContainerDied","Data":"7ac4ec48076907a3f6fda83cd5a22ecb76c972e6d23000df37de0848fb521c7f"}
Jul 18 01:31:56.143903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:56.143396    2567 scope.go:117] "RemoveContainer" containerID="b16a1a0cde99bd069e2e54dc09848c88263f4a0a80ba8bd58cee516f00e516b1"
Jul 18 01:31:56.165588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:56.165561    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"]
Jul 18 01:31:56.167599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:56.167580    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-gx6d8"]
Jul 18 01:31:56.918917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:56.918883    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" path="/var/lib/kubelet/pods/b8950684-09de-4b8d-a40f-9e5e21227152/volumes"
Jul 18 01:31:59.287285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287234    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"]
Jul 18 01:31:59.287676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287637    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287648    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287656    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287661    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287670    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287675    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287742    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34d6320-e040-4ef6-bd55-f06cc7ea3d46" containerName="revoke-keys"
Jul 18 01:31:59.287869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287751    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.287869 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.287757    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:31:59.291923 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.291904    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:31:59.297336 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.297302    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"]
Jul 18 01:31:59.415815 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.415775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8b45ff-c374-4fe5-bce9-40555ba97aed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2\" (UID: \"1e8b45ff-c374-4fe5-bce9-40555ba97aed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:31:59.516324 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.516285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8b45ff-c374-4fe5-bce9-40555ba97aed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2\" (UID: \"1e8b45ff-c374-4fe5-bce9-40555ba97aed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:31:59.516862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.516840    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8b45ff-c374-4fe5-bce9-40555ba97aed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2\" (UID: \"1e8b45ff-c374-4fe5-bce9-40555ba97aed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:31:59.604254 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.604219    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:31:59.749108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:31:59.749081    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"]
Jul 18 01:31:59.751758 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:31:59.751718    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8b45ff_c374_4fe5_bce9_40555ba97aed.slice/crio-d1909f40c67f3f2e4c01e3f08b5424982bb4c1e0f74f3346b6b0bb1597c9b4c8 WatchSource:0}: Error finding container d1909f40c67f3f2e4c01e3f08b5424982bb4c1e0f74f3346b6b0bb1597c9b4c8: Status 404 returned error can't find the container with id d1909f40c67f3f2e4c01e3f08b5424982bb4c1e0f74f3346b6b0bb1597c9b4c8
Jul 18 01:32:00.175991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:00.175955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerStarted","Data":"20f8b09ae8b2df5c6f1992d37ae531ba49bcc955cfa65ae6088238013676e3b7"}
Jul 18 01:32:00.176189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:00.176031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerStarted","Data":"d1909f40c67f3f2e4c01e3f08b5424982bb4c1e0f74f3346b6b0bb1597c9b4c8"}
Jul 18 01:32:00.191872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:00.191821    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" podStartSLOduration=1.191806748 podStartE2EDuration="1.191806748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:32:00.189646317 +0000 UTC m=+3099.717151278" watchObservedRunningTime="2026-07-18 01:32:00.191806748 +0000 UTC m=+3099.719311708"
Jul 18 01:32:01.183491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:01.183461    2567 generic.go:356] "Generic (PLEG): container finished" podID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerID="20f8b09ae8b2df5c6f1992d37ae531ba49bcc955cfa65ae6088238013676e3b7" exitCode=6
Jul 18 01:32:01.183859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:01.183505    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerDied","Data":"20f8b09ae8b2df5c6f1992d37ae531ba49bcc955cfa65ae6088238013676e3b7"}
Jul 18 01:32:01.183859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:01.183777    2567 scope.go:117] "RemoveContainer" containerID="20f8b09ae8b2df5c6f1992d37ae531ba49bcc955cfa65ae6088238013676e3b7"
Jul 18 01:32:02.191420 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:02.191375    2567 generic.go:356] "Generic (PLEG): container finished" podID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerID="5159a01edee38cd49bbc62d337bb9b6b1e4d4d58dd8654dd2a753565fc30a7a9" exitCode=6
Jul 18 01:32:02.191887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:02.191460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerDied","Data":"5159a01edee38cd49bbc62d337bb9b6b1e4d4d58dd8654dd2a753565fc30a7a9"}
Jul 18 01:32:02.191887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:02.191515    2567 scope.go:117] "RemoveContainer" containerID="20f8b09ae8b2df5c6f1992d37ae531ba49bcc955cfa65ae6088238013676e3b7"
Jul 18 01:32:02.191887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:02.191617    2567 scope.go:117] "RemoveContainer" containerID="5159a01edee38cd49bbc62d337bb9b6b1e4d4d58dd8654dd2a753565fc30a7a9"
Jul 18 01:32:02.192015 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:02.191898    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2_odh-ai-gateway-infra(1e8b45ff-c374-4fe5-bce9-40555ba97aed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed"
Jul 18 01:32:03.198556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:03.198526    2567 scope.go:117] "RemoveContainer" containerID="5159a01edee38cd49bbc62d337bb9b6b1e4d4d58dd8654dd2a753565fc30a7a9"
Jul 18 01:32:03.198975 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:03.198739    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2_odh-ai-gateway-infra(1e8b45ff-c374-4fe5-bce9-40555ba97aed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed"
Jul 18 01:32:09.913428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:09.913394    2567 scope.go:117] "RemoveContainer" containerID="388459fec11773aa3cf170ec0cc9084b6b525b3946fe3f747630bec290a06713"
Jul 18 01:32:10.239689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:10.239605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" event={"ID":"a1e6928b-71a3-4944-9d62-88402e6ff054","Type":"ContainerStarted","Data":"2bceb5b22453bce2cff51ec245448a8f7dbc5085e54ea227a7a291355426becc"}
Jul 18 01:32:10.255710 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:10.255655    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" podStartSLOduration=19.255638998 podStartE2EDuration="19.255638998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:32:10.253472324 +0000 UTC m=+3109.780977284" watchObservedRunningTime="2026-07-18 01:32:10.255638998 +0000 UTC m=+3109.783143965"
Jul 18 01:32:10.535987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:10.535809    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:32:11.246715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:11.246682    2567 generic.go:356] "Generic (PLEG): container finished" podID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerID="2bceb5b22453bce2cff51ec245448a8f7dbc5085e54ea227a7a291355426becc" exitCode=6
Jul 18 01:32:11.247219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:11.246760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" event={"ID":"a1e6928b-71a3-4944-9d62-88402e6ff054","Type":"ContainerDied","Data":"2bceb5b22453bce2cff51ec245448a8f7dbc5085e54ea227a7a291355426becc"}
Jul 18 01:32:11.247219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:11.246835    2567 scope.go:117] "RemoveContainer" containerID="388459fec11773aa3cf170ec0cc9084b6b525b3946fe3f747630bec290a06713"
Jul 18 01:32:11.247219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:11.247204    2567 scope.go:117] "RemoveContainer" containerID="2bceb5b22453bce2cff51ec245448a8f7dbc5085e54ea227a7a291355426becc"
Jul 18 01:32:11.247507 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:11.247485    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8_odh-ai-gateway-infra(a1e6928b-71a3-4944-9d62-88402e6ff054)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054"
Jul 18 01:32:11.263422 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:11.263394    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"]
Jul 18 01:32:12.419716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:12.419690    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:32:12.538354 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:12.538218    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1e6928b-71a3-4944-9d62-88402e6ff054-maas-api-service-ca\") pod \"a1e6928b-71a3-4944-9d62-88402e6ff054\" (UID: \"a1e6928b-71a3-4944-9d62-88402e6ff054\") "
Jul 18 01:32:12.538624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:12.538596    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1e6928b-71a3-4944-9d62-88402e6ff054-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1e6928b-71a3-4944-9d62-88402e6ff054" (UID: "a1e6928b-71a3-4944-9d62-88402e6ff054"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:32:12.640031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:12.639984    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1e6928b-71a3-4944-9d62-88402e6ff054-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:32:13.263136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:13.263036    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"
Jul 18 01:32:13.263136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:13.263062    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8" event={"ID":"a1e6928b-71a3-4944-9d62-88402e6ff054","Type":"ContainerDied","Data":"f6140ca6769a4794b3ad11df2b3be7e11ca13e43155acaea7e2c33df61dc239c"}
Jul 18 01:32:13.263420 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:13.263140    2567 scope.go:117] "RemoveContainer" containerID="2bceb5b22453bce2cff51ec245448a8f7dbc5085e54ea227a7a291355426becc"
Jul 18 01:32:13.278682 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:13.278655    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"]
Jul 18 01:32:13.285917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:13.285894    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-jq6f8"]
Jul 18 01:32:14.315407 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:14.315372    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:32:14.920159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:14.920123    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" path="/var/lib/kubelet/pods/a1e6928b-71a3-4944-9d62-88402e6ff054/volumes"
Jul 18 01:32:16.504235 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504199    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"]
Jul 18 01:32:16.504820 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504797    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.504820 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504818    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.504947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504856    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.504947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504866    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.505045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504966    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8950684-09de-4b8d-a40f-9e5e21227152" containerName="revoke-keys"
Jul 18 01:32:16.505045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504979    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.505045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.504994    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.505045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.505003    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:16.509467 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.509440    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:16.514393 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.514361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"]
Jul 18 01:32:16.580475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.580439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5863eba-a4f5-4995-a3b9-a2e60db0bec7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s\" (UID: \"d5863eba-a4f5-4995-a3b9-a2e60db0bec7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:16.681633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.681594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5863eba-a4f5-4995-a3b9-a2e60db0bec7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s\" (UID: \"d5863eba-a4f5-4995-a3b9-a2e60db0bec7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:16.682247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.682228    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5863eba-a4f5-4995-a3b9-a2e60db0bec7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s\" (UID: \"d5863eba-a4f5-4995-a3b9-a2e60db0bec7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:16.822830 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.822797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:16.978519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:16.978493    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"]
Jul 18 01:32:17.288031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:17.287979    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerStarted","Data":"a35077be0eb0ffc34c7dd9939deaa476e529384065cf2732f82e9a543cb6828b"}
Jul 18 01:32:17.288230 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:17.288041    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerStarted","Data":"4a4b6dfb11329cff943fc3d19cce2242636da6772c078d81a27df7cbe7b2056d"}
Jul 18 01:32:17.303458 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:17.303409    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" podStartSLOduration=1.30339367 podStartE2EDuration="1.30339367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:32:17.302159412 +0000 UTC m=+3116.829664374" watchObservedRunningTime="2026-07-18 01:32:17.30339367 +0000 UTC m=+3116.830898630"
Jul 18 01:32:17.913038 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:17.912950    2567 scope.go:117] "RemoveContainer" containerID="5159a01edee38cd49bbc62d337bb9b6b1e4d4d58dd8654dd2a753565fc30a7a9"
Jul 18 01:32:18.296607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:18.296513    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerStarted","Data":"3c656730107ac64e50f513f6ee9bf0fdb70de07732e474bd0fecf2a12ddb71d1"}
Jul 18 01:32:18.297947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:18.297918    2567 generic.go:356] "Generic (PLEG): container finished" podID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerID="a35077be0eb0ffc34c7dd9939deaa476e529384065cf2732f82e9a543cb6828b" exitCode=6
Jul 18 01:32:18.298073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:18.298002    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerDied","Data":"a35077be0eb0ffc34c7dd9939deaa476e529384065cf2732f82e9a543cb6828b"}
Jul 18 01:32:18.298149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:18.298136    2567 scope.go:117] "RemoveContainer" containerID="a35077be0eb0ffc34c7dd9939deaa476e529384065cf2732f82e9a543cb6828b"
Jul 18 01:32:19.306597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.306562    2567 generic.go:356] "Generic (PLEG): container finished" podID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerID="3c656730107ac64e50f513f6ee9bf0fdb70de07732e474bd0fecf2a12ddb71d1" exitCode=6
Jul 18 01:32:19.307017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.306646    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerDied","Data":"3c656730107ac64e50f513f6ee9bf0fdb70de07732e474bd0fecf2a12ddb71d1"}
Jul 18 01:32:19.307017 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.306728    2567 scope.go:117] "RemoveContainer" containerID="5159a01edee38cd49bbc62d337bb9b6b1e4d4d58dd8654dd2a753565fc30a7a9"
Jul 18 01:32:19.307124 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.307029    2567 scope.go:117] "RemoveContainer" containerID="3c656730107ac64e50f513f6ee9bf0fdb70de07732e474bd0fecf2a12ddb71d1"
Jul 18 01:32:19.307335 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:19.307303    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2_odh-ai-gateway-infra(1e8b45ff-c374-4fe5-bce9-40555ba97aed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed"
Jul 18 01:32:19.312353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.311869    2567 generic.go:356] "Generic (PLEG): container finished" podID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerID="e70f461c60f0e87cd03c251eff95f1163f6c1687abff2f9fd9ce8a6cd6892129" exitCode=6
Jul 18 01:32:19.312353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.311915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerDied","Data":"e70f461c60f0e87cd03c251eff95f1163f6c1687abff2f9fd9ce8a6cd6892129"}
Jul 18 01:32:19.312353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.312147    2567 scope.go:117] "RemoveContainer" containerID="e70f461c60f0e87cd03c251eff95f1163f6c1687abff2f9fd9ce8a6cd6892129"
Jul 18 01:32:19.312581 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:19.312376    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s_odh-ai-gateway-infra(d5863eba-a4f5-4995-a3b9-a2e60db0bec7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7"
Jul 18 01:32:19.319434 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.319417    2567 scope.go:117] "RemoveContainer" containerID="a35077be0eb0ffc34c7dd9939deaa476e529384065cf2732f82e9a543cb6828b"
Jul 18 01:32:19.322618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:19.322596    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"]
Jul 18 01:32:20.320935 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:20.320911    2567 scope.go:117] "RemoveContainer" containerID="e70f461c60f0e87cd03c251eff95f1163f6c1687abff2f9fd9ce8a6cd6892129"
Jul 18 01:32:20.321252 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:20.321096    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s_odh-ai-gateway-infra(d5863eba-a4f5-4995-a3b9-a2e60db0bec7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7"
Jul 18 01:32:20.460580 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:20.460556    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:32:20.617870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:20.617775    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8b45ff-c374-4fe5-bce9-40555ba97aed-maas-api-service-ca\") pod \"1e8b45ff-c374-4fe5-bce9-40555ba97aed\" (UID: \"1e8b45ff-c374-4fe5-bce9-40555ba97aed\") "
Jul 18 01:32:20.618139 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:20.618117    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8b45ff-c374-4fe5-bce9-40555ba97aed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e8b45ff-c374-4fe5-bce9-40555ba97aed" (UID: "1e8b45ff-c374-4fe5-bce9-40555ba97aed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:32:20.718809 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:20.718772    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8b45ff-c374-4fe5-bce9-40555ba97aed-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:32:21.328222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:21.328190    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"
Jul 18 01:32:21.328222 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:21.328219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2" event={"ID":"1e8b45ff-c374-4fe5-bce9-40555ba97aed","Type":"ContainerDied","Data":"d1909f40c67f3f2e4c01e3f08b5424982bb4c1e0f74f3346b6b0bb1597c9b4c8"}
Jul 18 01:32:21.328753 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:21.328331    2567 scope.go:117] "RemoveContainer" containerID="3c656730107ac64e50f513f6ee9bf0fdb70de07732e474bd0fecf2a12ddb71d1"
Jul 18 01:32:21.344056 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:21.344028    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"]
Jul 18 01:32:21.349461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:21.349438    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-nf2w2"]
Jul 18 01:32:22.919477 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:22.919442    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" path="/var/lib/kubelet/pods/1e8b45ff-c374-4fe5-bce9-40555ba97aed/volumes"
Jul 18 01:32:32.913718 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:32.913686    2567 scope.go:117] "RemoveContainer" containerID="e70f461c60f0e87cd03c251eff95f1163f6c1687abff2f9fd9ce8a6cd6892129"
Jul 18 01:32:33.396470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.396433    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerStarted","Data":"770d10cfea138647b796ab1ccd47e3dde561604a63d5ad30fb2c0f979faa565a"}
Jul 18 01:32:33.906474 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906391    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"]
Jul 18 01:32:33.906823 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906810    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:33.906866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906825    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e6928b-71a3-4944-9d62-88402e6ff054" containerName="revoke-keys"
Jul 18 01:32:33.906866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906840    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.906866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906846    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.906866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906852    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.906866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906858    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.906866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906866    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.907121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906872    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.907121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906938    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.907121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906945    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.907121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.906951    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8b45ff-c374-4fe5-bce9-40555ba97aed" containerName="revoke-keys"
Jul 18 01:32:33.910043 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.910025    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:33.917157 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.917127    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"]
Jul 18 01:32:33.939149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:33.939114    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7dd675d-27cd-403d-bcad-68cea5926a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n\" (UID: \"e7dd675d-27cd-403d-bcad-68cea5926a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:34.040320 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.040252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7dd675d-27cd-403d-bcad-68cea5926a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n\" (UID: \"e7dd675d-27cd-403d-bcad-68cea5926a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:34.040926 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.040898    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7dd675d-27cd-403d-bcad-68cea5926a09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n\" (UID: \"e7dd675d-27cd-403d-bcad-68cea5926a09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:34.222756 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.222652    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:34.403881 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.403848    2567 generic.go:356] "Generic (PLEG): container finished" podID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerID="770d10cfea138647b796ab1ccd47e3dde561604a63d5ad30fb2c0f979faa565a" exitCode=6
Jul 18 01:32:34.404062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.403901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerDied","Data":"770d10cfea138647b796ab1ccd47e3dde561604a63d5ad30fb2c0f979faa565a"}
Jul 18 01:32:34.404062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.403977    2567 scope.go:117] "RemoveContainer" containerID="e70f461c60f0e87cd03c251eff95f1163f6c1687abff2f9fd9ce8a6cd6892129"
Jul 18 01:32:34.404327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.404312    2567 scope.go:117] "RemoveContainer" containerID="770d10cfea138647b796ab1ccd47e3dde561604a63d5ad30fb2c0f979faa565a"
Jul 18 01:32:34.404582 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:34.404555    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s_odh-ai-gateway-infra(d5863eba-a4f5-4995-a3b9-a2e60db0bec7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7"
Jul 18 01:32:34.427589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.427556    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"]
Jul 18 01:32:34.571871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:34.571841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"]
Jul 18 01:32:34.573548 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:32:34.573520    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7dd675d_27cd_403d_bcad_68cea5926a09.slice/crio-d4a892f221db4d90aad491b1cbf46e531004f9ef6ce06d5f6e3225ac065c779d WatchSource:0}: Error finding container d4a892f221db4d90aad491b1cbf46e531004f9ef6ce06d5f6e3225ac065c779d: Status 404 returned error can't find the container with id d4a892f221db4d90aad491b1cbf46e531004f9ef6ce06d5f6e3225ac065c779d
Jul 18 01:32:35.411600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.411501    2567 generic.go:356] "Generic (PLEG): container finished" podID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerID="8e314a438b5fb7e5be2c166d6bdc20517624ad8e5d787391cd9256311ffb1dfa" exitCode=6
Jul 18 01:32:35.412061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.411589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" event={"ID":"e7dd675d-27cd-403d-bcad-68cea5926a09","Type":"ContainerDied","Data":"8e314a438b5fb7e5be2c166d6bdc20517624ad8e5d787391cd9256311ffb1dfa"}
Jul 18 01:32:35.412061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.411649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" event={"ID":"e7dd675d-27cd-403d-bcad-68cea5926a09","Type":"ContainerStarted","Data":"d4a892f221db4d90aad491b1cbf46e531004f9ef6ce06d5f6e3225ac065c779d"}
Jul 18 01:32:35.412061 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.411843    2567 scope.go:117] "RemoveContainer" containerID="8e314a438b5fb7e5be2c166d6bdc20517624ad8e5d787391cd9256311ffb1dfa"
Jul 18 01:32:35.583606 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.583574    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:35.656924 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.656888    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5863eba-a4f5-4995-a3b9-a2e60db0bec7-maas-api-service-ca\") pod \"d5863eba-a4f5-4995-a3b9-a2e60db0bec7\" (UID: \"d5863eba-a4f5-4995-a3b9-a2e60db0bec7\") "
Jul 18 01:32:35.657247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.657227    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5863eba-a4f5-4995-a3b9-a2e60db0bec7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5863eba-a4f5-4995-a3b9-a2e60db0bec7" (UID: "d5863eba-a4f5-4995-a3b9-a2e60db0bec7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:32:35.758111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:35.758086    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5863eba-a4f5-4995-a3b9-a2e60db0bec7-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:32:36.419822 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.419744    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"
Jul 18 01:32:36.419822 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.419790    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s" event={"ID":"d5863eba-a4f5-4995-a3b9-a2e60db0bec7","Type":"ContainerDied","Data":"4a4b6dfb11329cff943fc3d19cce2242636da6772c078d81a27df7cbe7b2056d"}
Jul 18 01:32:36.420319 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.419852    2567 scope.go:117] "RemoveContainer" containerID="770d10cfea138647b796ab1ccd47e3dde561604a63d5ad30fb2c0f979faa565a"
Jul 18 01:32:36.421985 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.421964    2567 generic.go:356] "Generic (PLEG): container finished" podID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerID="85ebfdf33b669402a70ed2f799604143760db57d636a3fec7966c4414be0f542" exitCode=6
Jul 18 01:32:36.422105 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.422001    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" event={"ID":"e7dd675d-27cd-403d-bcad-68cea5926a09","Type":"ContainerDied","Data":"85ebfdf33b669402a70ed2f799604143760db57d636a3fec7966c4414be0f542"}
Jul 18 01:32:36.422361 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.422342    2567 scope.go:117] "RemoveContainer" containerID="85ebfdf33b669402a70ed2f799604143760db57d636a3fec7966c4414be0f542"
Jul 18 01:32:36.422636 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:36.422605    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n_odh-ai-gateway-infra(e7dd675d-27cd-403d-bcad-68cea5926a09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09"
Jul 18 01:32:36.429842 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.429821    2567 scope.go:117] "RemoveContainer" containerID="8e314a438b5fb7e5be2c166d6bdc20517624ad8e5d787391cd9256311ffb1dfa"
Jul 18 01:32:36.447725 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.447702    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"]
Jul 18 01:32:36.454045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.454016    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-9xc9s"]
Jul 18 01:32:36.919456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:36.919421    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" path="/var/lib/kubelet/pods/d5863eba-a4f5-4995-a3b9-a2e60db0bec7/volumes"
Jul 18 01:32:37.428795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:37.428768    2567 scope.go:117] "RemoveContainer" containerID="85ebfdf33b669402a70ed2f799604143760db57d636a3fec7966c4414be0f542"
Jul 18 01:32:37.429247 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:37.429006    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n_odh-ai-gateway-infra(e7dd675d-27cd-403d-bcad-68cea5926a09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09"
Jul 18 01:32:38.611373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:38.611336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:32:50.916454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:50.916377    2567 scope.go:117] "RemoveContainer" containerID="85ebfdf33b669402a70ed2f799604143760db57d636a3fec7966c4414be0f542"
Jul 18 01:32:51.489607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.489519    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"]
Jul 18 01:32:51.489956 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.489942    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:51.490000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.489958    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:51.490000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.489981    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:51.490000 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.489987    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:51.490097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.490047    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:51.490097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.490058    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:51.493301 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.493279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:32:51.500211 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.500183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"]
Jul 18 01:32:51.515301 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.515229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" event={"ID":"e7dd675d-27cd-403d-bcad-68cea5926a09","Type":"ContainerStarted","Data":"9aec560767467d518f73f95004a585d110943358a7ccb40348d37fd76dc99621"}
Jul 18 01:32:51.531806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.531743    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" podStartSLOduration=18.531727828 podStartE2EDuration="18.531727828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:32:51.530150129 +0000 UTC m=+3151.057655090" watchObservedRunningTime="2026-07-18 01:32:51.531727828 +0000 UTC m=+3151.059232787"
Jul 18 01:32:51.605570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.605531    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz\" (UID: \"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:32:51.706773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.706729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz\" (UID: \"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:32:51.707394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.707371    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz\" (UID: \"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:32:51.805517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.805441    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:32:51.957187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:51.957163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"]
Jul 18 01:32:51.959690 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:32:51.959653    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3904a3a1_ddb4_43a5_a064_9015cd3e7b9c.slice/crio-7672eb8c7fc4c0dcca44bfb6e6bfd1edbbba6b2908d06e7c2ef1e07767fc47a8 WatchSource:0}: Error finding container 7672eb8c7fc4c0dcca44bfb6e6bfd1edbbba6b2908d06e7c2ef1e07767fc47a8: Status 404 returned error can't find the container with id 7672eb8c7fc4c0dcca44bfb6e6bfd1edbbba6b2908d06e7c2ef1e07767fc47a8
Jul 18 01:32:52.522930 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.522826    2567 generic.go:356] "Generic (PLEG): container finished" podID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerID="9aec560767467d518f73f95004a585d110943358a7ccb40348d37fd76dc99621" exitCode=6
Jul 18 01:32:52.522930 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.522901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" event={"ID":"e7dd675d-27cd-403d-bcad-68cea5926a09","Type":"ContainerDied","Data":"9aec560767467d518f73f95004a585d110943358a7ccb40348d37fd76dc99621"}
Jul 18 01:32:52.523165 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.523003    2567 scope.go:117] "RemoveContainer" containerID="85ebfdf33b669402a70ed2f799604143760db57d636a3fec7966c4414be0f542"
Jul 18 01:32:52.523390 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.523369    2567 scope.go:117] "RemoveContainer" containerID="9aec560767467d518f73f95004a585d110943358a7ccb40348d37fd76dc99621"
Jul 18 01:32:52.523655 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:52.523625    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n_odh-ai-gateway-infra(e7dd675d-27cd-403d-bcad-68cea5926a09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09"
Jul 18 01:32:52.524671 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.524642    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerStarted","Data":"1c116a5e9d89c7f209549080dc5bf243fa97e5a82c9690b2bab0b18944adbae2"}
Jul 18 01:32:52.524780 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.524697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerStarted","Data":"7672eb8c7fc4c0dcca44bfb6e6bfd1edbbba6b2908d06e7c2ef1e07767fc47a8"}
Jul 18 01:32:52.545478 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.545448    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"]
Jul 18 01:32:52.559229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:52.559172    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" podStartSLOduration=1.559156879 podStartE2EDuration="1.559156879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:32:52.556630061 +0000 UTC m=+3152.084135022" watchObservedRunningTime="2026-07-18 01:32:52.559156879 +0000 UTC m=+3152.086661839"
Jul 18 01:32:53.533302 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.533249    2567 generic.go:356] "Generic (PLEG): container finished" podID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerID="1c116a5e9d89c7f209549080dc5bf243fa97e5a82c9690b2bab0b18944adbae2" exitCode=6
Jul 18 01:32:53.533620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.533311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerDied","Data":"1c116a5e9d89c7f209549080dc5bf243fa97e5a82c9690b2bab0b18944adbae2"}
Jul 18 01:32:53.533620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.533606    2567 scope.go:117] "RemoveContainer" containerID="1c116a5e9d89c7f209549080dc5bf243fa97e5a82c9690b2bab0b18944adbae2"
Jul 18 01:32:53.688316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.688291    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:53.831471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.831433    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7dd675d-27cd-403d-bcad-68cea5926a09-maas-api-service-ca\") pod \"e7dd675d-27cd-403d-bcad-68cea5926a09\" (UID: \"e7dd675d-27cd-403d-bcad-68cea5926a09\") "
Jul 18 01:32:53.831810 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.831785    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7dd675d-27cd-403d-bcad-68cea5926a09-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e7dd675d-27cd-403d-bcad-68cea5926a09" (UID: "e7dd675d-27cd-403d-bcad-68cea5926a09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:32:53.932847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:53.932805    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7dd675d-27cd-403d-bcad-68cea5926a09-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:32:54.540808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.540712    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"
Jul 18 01:32:54.540808 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.540716    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n" event={"ID":"e7dd675d-27cd-403d-bcad-68cea5926a09","Type":"ContainerDied","Data":"d4a892f221db4d90aad491b1cbf46e531004f9ef6ce06d5f6e3225ac065c779d"}
Jul 18 01:32:54.541378 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.540874    2567 scope.go:117] "RemoveContainer" containerID="9aec560767467d518f73f95004a585d110943358a7ccb40348d37fd76dc99621"
Jul 18 01:32:54.542896 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.542866    2567 generic.go:356] "Generic (PLEG): container finished" podID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerID="aca67e50c64b884c46370d9ebd740ed7948ff8f008f5313ff4b0783b904e991f" exitCode=6
Jul 18 01:32:54.542999 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.542936    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerDied","Data":"aca67e50c64b884c46370d9ebd740ed7948ff8f008f5313ff4b0783b904e991f"}
Jul 18 01:32:54.543190 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.543174    2567 scope.go:117] "RemoveContainer" containerID="aca67e50c64b884c46370d9ebd740ed7948ff8f008f5313ff4b0783b904e991f"
Jul 18 01:32:54.543491 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:54.543469    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz_odh-ai-gateway-infra(3904a3a1-ddb4-43a5-a064-9015cd3e7b9c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c"
Jul 18 01:32:54.550934 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.550915    2567 scope.go:117] "RemoveContainer" containerID="1c116a5e9d89c7f209549080dc5bf243fa97e5a82c9690b2bab0b18944adbae2"
Jul 18 01:32:54.572987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.572955    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"]
Jul 18 01:32:54.576898 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.576869    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zrv8n"]
Jul 18 01:32:54.919996 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:54.919959    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" path="/var/lib/kubelet/pods/e7dd675d-27cd-403d-bcad-68cea5926a09/volumes"
Jul 18 01:32:55.551172 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:55.551148    2567 scope.go:117] "RemoveContainer" containerID="aca67e50c64b884c46370d9ebd740ed7948ff8f008f5313ff4b0783b904e991f"
Jul 18 01:32:55.551611 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:32:55.551379    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz_odh-ai-gateway-infra(3904a3a1-ddb4-43a5-a064-9015cd3e7b9c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c"
Jul 18 01:32:57.208736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:57.208697    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:32:59.326792 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.326761    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"]
Jul 18 01:32:59.327198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327141    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.327198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327151    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.327198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327165    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.327198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327171    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.327198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327180    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:59.327198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327186    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:59.327439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327256    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.327439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327279    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.327439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327285    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5863eba-a4f5-4995-a3b9-a2e60db0bec7" containerName="revoke-keys"
Jul 18 01:32:59.327439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.327293    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:32:59.331904 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.331884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:32:59.339097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.339064    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"]
Jul 18 01:32:59.489342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.489258    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5f8af0-c061-41df-839e-3074f069d1b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4\" (UID: \"cc5f8af0-c061-41df-839e-3074f069d1b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:32:59.590440 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.590338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5f8af0-c061-41df-839e-3074f069d1b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4\" (UID: \"cc5f8af0-c061-41df-839e-3074f069d1b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:32:59.591004 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.590978    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5f8af0-c061-41df-839e-3074f069d1b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4\" (UID: \"cc5f8af0-c061-41df-839e-3074f069d1b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:32:59.644022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.643991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:32:59.784306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:32:59.784282    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"]
Jul 18 01:32:59.787216 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:32:59.787187    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5f8af0_c061_41df_839e_3074f069d1b5.slice/crio-e81a1287c478b90c0e7e8be7ad317ac0dad13265ab9239be0edc90a25551099a WatchSource:0}: Error finding container e81a1287c478b90c0e7e8be7ad317ac0dad13265ab9239be0edc90a25551099a: Status 404 returned error can't find the container with id e81a1287c478b90c0e7e8be7ad317ac0dad13265ab9239be0edc90a25551099a
Jul 18 01:33:00.582208 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:00.582171    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerID="e763b375111213bfc1c683e0f54c69cafff3cb2bbb4b13c72f0ea7b3b968ab51" exitCode=6
Jul 18 01:33:00.582626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:00.582259    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" event={"ID":"cc5f8af0-c061-41df-839e-3074f069d1b5","Type":"ContainerDied","Data":"e763b375111213bfc1c683e0f54c69cafff3cb2bbb4b13c72f0ea7b3b968ab51"}
Jul 18 01:33:00.582626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:00.582333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" event={"ID":"cc5f8af0-c061-41df-839e-3074f069d1b5","Type":"ContainerStarted","Data":"e81a1287c478b90c0e7e8be7ad317ac0dad13265ab9239be0edc90a25551099a"}
Jul 18 01:33:00.582626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:00.582425    2567 scope.go:117] "RemoveContainer" containerID="e763b375111213bfc1c683e0f54c69cafff3cb2bbb4b13c72f0ea7b3b968ab51"
Jul 18 01:33:01.589870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:01.589835    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerID="2f5df03362b18e05c00003f6990fc4486f0f53dc9081102912977b36200fc17e" exitCode=6
Jul 18 01:33:01.590371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:01.589915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" event={"ID":"cc5f8af0-c061-41df-839e-3074f069d1b5","Type":"ContainerDied","Data":"2f5df03362b18e05c00003f6990fc4486f0f53dc9081102912977b36200fc17e"}
Jul 18 01:33:01.590371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:01.589984    2567 scope.go:117] "RemoveContainer" containerID="e763b375111213bfc1c683e0f54c69cafff3cb2bbb4b13c72f0ea7b3b968ab51"
Jul 18 01:33:01.590371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:01.590190    2567 scope.go:117] "RemoveContainer" containerID="2f5df03362b18e05c00003f6990fc4486f0f53dc9081102912977b36200fc17e"
Jul 18 01:33:01.590548 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:01.590501    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4_odh-ai-gateway-infra(cc5f8af0-c061-41df-839e-3074f069d1b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5"
Jul 18 01:33:02.600338 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:02.600315    2567 scope.go:117] "RemoveContainer" containerID="2f5df03362b18e05c00003f6990fc4486f0f53dc9081102912977b36200fc17e"
Jul 18 01:33:02.600733 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:02.600524    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4_odh-ai-gateway-infra(cc5f8af0-c061-41df-839e-3074f069d1b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5"
Jul 18 01:33:06.349559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.349523    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6cv4d"]
Jul 18 01:33:06.350011 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.349994    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:33:06.350011 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.350013    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd675d-27cd-403d-bcad-68cea5926a09" containerName="revoke-keys"
Jul 18 01:33:06.354065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.354040    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:06.362308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.362258    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6cv4d"]
Jul 18 01:33:06.459509 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.459473    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfj9z\" (UniqueName: \"kubernetes.io/projected/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1-kube-api-access-cfj9z\") pod \"kuadrant-operator-catalog-6cv4d\" (UID: \"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1\") " pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:06.549858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.549820    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"]
Jul 18 01:33:06.553487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.553463    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:06.560192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.560163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"]
Jul 18 01:33:06.560461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.560432    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfj9z\" (UniqueName: \"kubernetes.io/projected/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1-kube-api-access-cfj9z\") pod \"kuadrant-operator-catalog-6cv4d\" (UID: \"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1\") " pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:06.571429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.571404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfj9z\" (UniqueName: \"kubernetes.io/projected/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1-kube-api-access-cfj9z\") pod \"kuadrant-operator-catalog-6cv4d\" (UID: \"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1\") " pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:06.661224 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.661119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fead149-4886-4ddd-bec4-ec6fef667976-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt\" (UID: \"9fead149-4886-4ddd-bec4-ec6fef667976\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:06.666480 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.666437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:06.763083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.762053    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fead149-4886-4ddd-bec4-ec6fef667976-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt\" (UID: \"9fead149-4886-4ddd-bec4-ec6fef667976\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:06.763083 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.763005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fead149-4886-4ddd-bec4-ec6fef667976-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt\" (UID: \"9fead149-4886-4ddd-bec4-ec6fef667976\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:06.810171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.810139    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6cv4d"]
Jul 18 01:33:06.811407 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:33:06.811383    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef534b5d_e684_4bf7_9b7c_2e9c2f43b7d1.slice/crio-2c914c5a711b399b0354a9d8c4bf5141059f4de4320426e02721521a36bc10fb WatchSource:0}: Error finding container 2c914c5a711b399b0354a9d8c4bf5141059f4de4320426e02721521a36bc10fb: Status 404 returned error can't find the container with id 2c914c5a711b399b0354a9d8c4bf5141059f4de4320426e02721521a36bc10fb
Jul 18 01:33:06.812832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.812817    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:33:06.865591 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:06.865564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:07.009020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.008993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"]
Jul 18 01:33:07.010687 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:33:07.010660    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fead149_4886_4ddd_bec4_ec6fef667976.slice/crio-c180326fe21588ddae641d0da992f2f436c1f4f85ff35341b1f3dd4d0ead9d6b WatchSource:0}: Error finding container c180326fe21588ddae641d0da992f2f436c1f4f85ff35341b1f3dd4d0ead9d6b: Status 404 returned error can't find the container with id c180326fe21588ddae641d0da992f2f436c1f4f85ff35341b1f3dd4d0ead9d6b
Jul 18 01:33:07.630226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.630185    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d" event={"ID":"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1","Type":"ContainerStarted","Data":"d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728"}
Jul 18 01:33:07.630689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.630250    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d" event={"ID":"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1","Type":"ContainerStarted","Data":"2c914c5a711b399b0354a9d8c4bf5141059f4de4320426e02721521a36bc10fb"}
Jul 18 01:33:07.631961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.631929    2567 generic.go:356] "Generic (PLEG): container finished" podID="9fead149-4886-4ddd-bec4-ec6fef667976" containerID="71e0ec837ab11030b9bbae1b1fcb08ff3c9e6441a02283c170ba3dda13e53872" exitCode=6
Jul 18 01:33:07.632123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.632090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" event={"ID":"9fead149-4886-4ddd-bec4-ec6fef667976","Type":"ContainerDied","Data":"71e0ec837ab11030b9bbae1b1fcb08ff3c9e6441a02283c170ba3dda13e53872"}
Jul 18 01:33:07.632182 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.632129    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" event={"ID":"9fead149-4886-4ddd-bec4-ec6fef667976","Type":"ContainerStarted","Data":"c180326fe21588ddae641d0da992f2f436c1f4f85ff35341b1f3dd4d0ead9d6b"}
Jul 18 01:33:07.632242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.632197    2567 scope.go:117] "RemoveContainer" containerID="71e0ec837ab11030b9bbae1b1fcb08ff3c9e6441a02283c170ba3dda13e53872"
Jul 18 01:33:07.646451 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:07.646405    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d" podStartSLOduration=1.27582519 podStartE2EDuration="1.646388106s" totalImagesPullingTime="370.562916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:33:07.645075591 +0000 UTC m=+3167.172580552" watchObservedRunningTime="2026-07-18 01:33:07.646388106 +0000 UTC m=+3167.173893066"
Jul 18 01:33:08.647437 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:08.647348    2567 generic.go:356] "Generic (PLEG): container finished" podID="9fead149-4886-4ddd-bec4-ec6fef667976" containerID="d951dfe965a372e3b5d65592abd16db9ca026d9ef65a72949c607d9a3a73c0a1" exitCode=6
Jul 18 01:33:08.647826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:08.647430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" event={"ID":"9fead149-4886-4ddd-bec4-ec6fef667976","Type":"ContainerDied","Data":"d951dfe965a372e3b5d65592abd16db9ca026d9ef65a72949c607d9a3a73c0a1"}
Jul 18 01:33:08.647826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:08.647523    2567 scope.go:117] "RemoveContainer" containerID="71e0ec837ab11030b9bbae1b1fcb08ff3c9e6441a02283c170ba3dda13e53872"
Jul 18 01:33:08.647826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:08.647720    2567 scope.go:117] "RemoveContainer" containerID="d951dfe965a372e3b5d65592abd16db9ca026d9ef65a72949c607d9a3a73c0a1"
Jul 18 01:33:08.647992 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:08.647934    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt_odh-ai-gateway-infra(9fead149-4886-4ddd-bec4-ec6fef667976)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" podUID="9fead149-4886-4ddd-bec4-ec6fef667976"
Jul 18 01:33:08.920095 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:08.920005    2567 scope.go:117] "RemoveContainer" containerID="aca67e50c64b884c46370d9ebd740ed7948ff8f008f5313ff4b0783b904e991f"
Jul 18 01:33:09.656111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:09.656074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerStarted","Data":"35c4348c15697e799495f89889a62f486a0b426a3510e8de61ade9b8b4cdf01d"}
Jul 18 01:33:09.658205 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:09.658185    2567 scope.go:117] "RemoveContainer" containerID="d951dfe965a372e3b5d65592abd16db9ca026d9ef65a72949c607d9a3a73c0a1"
Jul 18 01:33:09.658416 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:09.658397    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt_odh-ai-gateway-infra(9fead149-4886-4ddd-bec4-ec6fef667976)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" podUID="9fead149-4886-4ddd-bec4-ec6fef667976"
Jul 18 01:33:10.665639 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:10.665608    2567 generic.go:356] "Generic (PLEG): container finished" podID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerID="35c4348c15697e799495f89889a62f486a0b426a3510e8de61ade9b8b4cdf01d" exitCode=6
Jul 18 01:33:10.666108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:10.665682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerDied","Data":"35c4348c15697e799495f89889a62f486a0b426a3510e8de61ade9b8b4cdf01d"}
Jul 18 01:33:10.666108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:10.665745    2567 scope.go:117] "RemoveContainer" containerID="aca67e50c64b884c46370d9ebd740ed7948ff8f008f5313ff4b0783b904e991f"
Jul 18 01:33:10.666108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:10.666018    2567 scope.go:117] "RemoveContainer" containerID="35c4348c15697e799495f89889a62f486a0b426a3510e8de61ade9b8b4cdf01d"
Jul 18 01:33:10.666286 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:10.666245    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz_odh-ai-gateway-infra(3904a3a1-ddb4-43a5-a064-9015cd3e7b9c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c"
Jul 18 01:33:10.712747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:10.712716    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"]
Jul 18 01:33:11.823158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:11.823126    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:33:12.012731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.012635    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c-maas-api-service-ca\") pod \"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c\" (UID: \"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c\") "
Jul 18 01:33:12.013068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.013034    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" (UID: "3904a3a1-ddb4-43a5-a064-9015cd3e7b9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:33:12.114483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.114448    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:33:12.680365 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.680329    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"
Jul 18 01:33:12.680535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.680367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz" event={"ID":"3904a3a1-ddb4-43a5-a064-9015cd3e7b9c","Type":"ContainerDied","Data":"7672eb8c7fc4c0dcca44bfb6e6bfd1edbbba6b2908d06e7c2ef1e07767fc47a8"}
Jul 18 01:33:12.680535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.680429    2567 scope.go:117] "RemoveContainer" containerID="35c4348c15697e799495f89889a62f486a0b426a3510e8de61ade9b8b4cdf01d"
Jul 18 01:33:12.710021 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.709978    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"]
Jul 18 01:33:12.710187 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.710040    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-sqsdz"]
Jul 18 01:33:12.919407 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:12.919373    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" path="/var/lib/kubelet/pods/3904a3a1-ddb4-43a5-a064-9015cd3e7b9c/volumes"
Jul 18 01:33:13.503004 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:13.502968    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:33:15.913109 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:15.913079    2567 scope.go:117] "RemoveContainer" containerID="2f5df03362b18e05c00003f6990fc4486f0f53dc9081102912977b36200fc17e"
Jul 18 01:33:16.667032 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.666991    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:16.667262 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.667068    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:16.691098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.691072    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:16.705637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.705603    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerID="329528bafb9a1a2c512e6c45666d1a0a8d7f46a2113ca568813dc6ec6571fece" exitCode=6
Jul 18 01:33:16.705950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.705680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" event={"ID":"cc5f8af0-c061-41df-839e-3074f069d1b5","Type":"ContainerDied","Data":"329528bafb9a1a2c512e6c45666d1a0a8d7f46a2113ca568813dc6ec6571fece"}
Jul 18 01:33:16.705950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.705768    2567 scope.go:117] "RemoveContainer" containerID="2f5df03362b18e05c00003f6990fc4486f0f53dc9081102912977b36200fc17e"
Jul 18 01:33:16.706283 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.706253    2567 scope.go:117] "RemoveContainer" containerID="329528bafb9a1a2c512e6c45666d1a0a8d7f46a2113ca568813dc6ec6571fece"
Jul 18 01:33:16.706512 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:16.706489    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4_odh-ai-gateway-infra(cc5f8af0-c061-41df-839e-3074f069d1b5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5"
Jul 18 01:33:16.731578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.731550    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:16.919550 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:16.919468    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6cv4d"]
Jul 18 01:33:17.730939 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:17.730893    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"]
Jul 18 01:33:17.881845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:17.881820    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:33:17.967657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:17.967623    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5f8af0-c061-41df-839e-3074f069d1b5-maas-api-service-ca\") pod \"cc5f8af0-c061-41df-839e-3074f069d1b5\" (UID: \"cc5f8af0-c061-41df-839e-3074f069d1b5\") "
Jul 18 01:33:17.968094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:17.967994    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5f8af0-c061-41df-839e-3074f069d1b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc5f8af0-c061-41df-839e-3074f069d1b5" (UID: "cc5f8af0-c061-41df-839e-3074f069d1b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:33:18.068952 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.068915    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5f8af0-c061-41df-839e-3074f069d1b5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:33:18.720131 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.720101    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"
Jul 18 01:33:18.720371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.720146    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4" event={"ID":"cc5f8af0-c061-41df-839e-3074f069d1b5","Type":"ContainerDied","Data":"e81a1287c478b90c0e7e8be7ad317ac0dad13265ab9239be0edc90a25551099a"}
Jul 18 01:33:18.720371 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.720218    2567 scope.go:117] "RemoveContainer" containerID="329528bafb9a1a2c512e6c45666d1a0a8d7f46a2113ca568813dc6ec6571fece"
Jul 18 01:33:18.720519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.720384    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d" podUID="ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" containerName="registry-server" containerID="cri-o://d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728" gracePeriod=2
Jul 18 01:33:18.751452 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.751423    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"]
Jul 18 01:33:18.754872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.754846    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-mtjh4"]
Jul 18 01:33:18.940924 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.940892    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" path="/var/lib/kubelet/pods/cc5f8af0-c061-41df-839e-3074f069d1b5/volumes"
Jul 18 01:33:18.998227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:18.998198    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:19.080129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.080091    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfj9z\" (UniqueName: \"kubernetes.io/projected/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1-kube-api-access-cfj9z\") pod \"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1\" (UID: \"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1\") "
Jul 18 01:33:19.082331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.082300    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1-kube-api-access-cfj9z" (OuterVolumeSpecName: "kube-api-access-cfj9z") pod "ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" (UID: "ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1"). InnerVolumeSpecName "kube-api-access-cfj9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:33:19.180908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.180869    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfj9z\" (UniqueName: \"kubernetes.io/projected/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1-kube-api-access-cfj9z\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:33:19.727792 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.727755    2567 generic.go:356] "Generic (PLEG): container finished" podID="ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" containerID="d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728" exitCode=0
Jul 18 01:33:19.727980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.727805    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d"
Jul 18 01:33:19.727980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.727841    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d" event={"ID":"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1","Type":"ContainerDied","Data":"d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728"}
Jul 18 01:33:19.727980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.727899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6cv4d" event={"ID":"ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1","Type":"ContainerDied","Data":"2c914c5a711b399b0354a9d8c4bf5141059f4de4320426e02721521a36bc10fb"}
Jul 18 01:33:19.727980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.727926    2567 scope.go:117] "RemoveContainer" containerID="d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728"
Jul 18 01:33:19.737920 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.737892    2567 scope.go:117] "RemoveContainer" containerID="d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728"
Jul 18 01:33:19.738192 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:19.738172    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728\": container with ID starting with d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728 not found: ID does not exist" containerID="d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728"
Jul 18 01:33:19.738234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.738202    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728"} err="failed to get container status \"d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728\": rpc error: code = NotFound desc = could not find container \"d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728\": container with ID starting with d934e7d998ae1cd24b6f49071a6ff1f3ad4c45afb46b898d79a99276551e3728 not found: ID does not exist"
Jul 18 01:33:19.748998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.748976    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6cv4d"]
Jul 18 01:33:19.752534 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:19.752513    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6cv4d"]
Jul 18 01:33:20.919349 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:20.919316    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" path="/var/lib/kubelet/pods/ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1/volumes"
Jul 18 01:33:24.919508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:24.919482    2567 scope.go:117] "RemoveContainer" containerID="d951dfe965a372e3b5d65592abd16db9ca026d9ef65a72949c607d9a3a73c0a1"
Jul 18 01:33:25.762225 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:25.762196    2567 generic.go:356] "Generic (PLEG): container finished" podID="9fead149-4886-4ddd-bec4-ec6fef667976" containerID="0b26011ea09db780c6055180917bfaf2159640753c4c50fea8f5de8e9f4f0c18" exitCode=6
Jul 18 01:33:25.762417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:25.762301    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" event={"ID":"9fead149-4886-4ddd-bec4-ec6fef667976","Type":"ContainerDied","Data":"0b26011ea09db780c6055180917bfaf2159640753c4c50fea8f5de8e9f4f0c18"}
Jul 18 01:33:25.762417 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:25.762373    2567 scope.go:117] "RemoveContainer" containerID="d951dfe965a372e3b5d65592abd16db9ca026d9ef65a72949c607d9a3a73c0a1"
Jul 18 01:33:25.762692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:25.762669    2567 scope.go:117] "RemoveContainer" containerID="0b26011ea09db780c6055180917bfaf2159640753c4c50fea8f5de8e9f4f0c18"
Jul 18 01:33:25.762947 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:25.762921    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt_odh-ai-gateway-infra(9fead149-4886-4ddd-bec4-ec6fef667976)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" podUID="9fead149-4886-4ddd-bec4-ec6fef667976"
Jul 18 01:33:26.789178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:26.789144    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"]
Jul 18 01:33:26.931923 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:26.931900    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:27.058294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.058257    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fead149-4886-4ddd-bec4-ec6fef667976-maas-api-service-ca\") pod \"9fead149-4886-4ddd-bec4-ec6fef667976\" (UID: \"9fead149-4886-4ddd-bec4-ec6fef667976\") "
Jul 18 01:33:27.058631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.058609    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fead149-4886-4ddd-bec4-ec6fef667976-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9fead149-4886-4ddd-bec4-ec6fef667976" (UID: "9fead149-4886-4ddd-bec4-ec6fef667976"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:33:27.159047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.158993    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fead149-4886-4ddd-bec4-ec6fef667976-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:33:27.776733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.776702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt" event={"ID":"9fead149-4886-4ddd-bec4-ec6fef667976","Type":"ContainerDied","Data":"c180326fe21588ddae641d0da992f2f436c1f4f85ff35341b1f3dd4d0ead9d6b"}
Jul 18 01:33:27.776733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.776727    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"
Jul 18 01:33:27.776995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.776764    2567 scope.go:117] "RemoveContainer" containerID="0b26011ea09db780c6055180917bfaf2159640753c4c50fea8f5de8e9f4f0c18"
Jul 18 01:33:27.799659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.799632    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"]
Jul 18 01:33:27.803686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:27.803665    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-6xxqt"]
Jul 18 01:33:28.919194 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:28.919148    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" path="/var/lib/kubelet/pods/9fead149-4886-4ddd-bec4-ec6fef667976/volumes"
Jul 18 01:33:31.005523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:31.005485    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:33:33.951519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951485    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"]
Jul 18 01:33:33.951921 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951906    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.951921 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951919    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951929    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951935    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951947    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951953    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951978    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951984    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951992    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" containerName="registry-server"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.951998    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" containerName="registry-server"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952003    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952007    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952015    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952020    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952025    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952032    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952046    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952055    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952111    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952120    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef534b5d-e684-4bf7-9b7c-2e9c2f43b7d1" containerName="registry-server"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952132    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952140    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952146    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952150    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952157    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fead149-4886-4ddd-bec4-ec6fef667976" containerName="revoke-keys"
Jul 18 01:33:33.952353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.952163    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:33.955260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.955242    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:33.957652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.957631    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:33:33.957769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.957660    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:33:33.962518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:33.962490    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"]
Jul 18 01:33:34.020346 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.020301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg\" (UID: \"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:34.121797 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.121739    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg\" (UID: \"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:34.122447 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.122423    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg\" (UID: \"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:34.267606 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.267520    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:34.408018 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.407983    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"]
Jul 18 01:33:34.409014 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:33:34.408985    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27687bf5_c4ec_45e8_aba9_cf3d9ac0ab30.slice/crio-12b679f3416e4accaef22a81bfe33fa19d6a41f99b354e95e0c50ebf352ebee8 WatchSource:0}: Error finding container 12b679f3416e4accaef22a81bfe33fa19d6a41f99b354e95e0c50ebf352ebee8: Status 404 returned error can't find the container with id 12b679f3416e4accaef22a81bfe33fa19d6a41f99b354e95e0c50ebf352ebee8
Jul 18 01:33:34.817054 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.817013    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" event={"ID":"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30","Type":"ContainerStarted","Data":"956542515c3c6353ac6ff3b65f05e8b59d054cefffda58de25f205a1822d9ae5"}
Jul 18 01:33:34.817260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.817076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" event={"ID":"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30","Type":"ContainerStarted","Data":"12b679f3416e4accaef22a81bfe33fa19d6a41f99b354e95e0c50ebf352ebee8"}
Jul 18 01:33:34.833183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:34.833129    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" podStartSLOduration=1.833112581 podStartE2EDuration="1.833112581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:33:34.831093371 +0000 UTC m=+3194.358598334" watchObservedRunningTime="2026-07-18 01:33:34.833112581 +0000 UTC m=+3194.360617541"
Jul 18 01:33:35.823711 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:35.823674    2567 generic.go:356] "Generic (PLEG): container finished" podID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerID="956542515c3c6353ac6ff3b65f05e8b59d054cefffda58de25f205a1822d9ae5" exitCode=6
Jul 18 01:33:35.824138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:35.823764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" event={"ID":"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30","Type":"ContainerDied","Data":"956542515c3c6353ac6ff3b65f05e8b59d054cefffda58de25f205a1822d9ae5"}
Jul 18 01:33:35.824138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:35.823923    2567 scope.go:117] "RemoveContainer" containerID="956542515c3c6353ac6ff3b65f05e8b59d054cefffda58de25f205a1822d9ae5"
Jul 18 01:33:36.831006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:36.830975    2567 generic.go:356] "Generic (PLEG): container finished" podID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerID="747a4f3b6472c054c8af6cd04c4038abae0d6a89eef59def23ddebad27f9f277" exitCode=6
Jul 18 01:33:36.831507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:36.831060    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" event={"ID":"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30","Type":"ContainerDied","Data":"747a4f3b6472c054c8af6cd04c4038abae0d6a89eef59def23ddebad27f9f277"}
Jul 18 01:33:36.831507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:36.831115    2567 scope.go:117] "RemoveContainer" containerID="956542515c3c6353ac6ff3b65f05e8b59d054cefffda58de25f205a1822d9ae5"
Jul 18 01:33:36.831507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:36.831360    2567 scope.go:117] "RemoveContainer" containerID="747a4f3b6472c054c8af6cd04c4038abae0d6a89eef59def23ddebad27f9f277"
Jul 18 01:33:36.831633 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:36.831582    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg_odh-ai-gateway-infra(27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30"
Jul 18 01:33:37.838143 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:37.838113    2567 scope.go:117] "RemoveContainer" containerID="747a4f3b6472c054c8af6cd04c4038abae0d6a89eef59def23ddebad27f9f277"
Jul 18 01:33:37.838542 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:37.838350    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg_odh-ai-gateway-infra(27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30"
Jul 18 01:33:45.304773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:45.304738    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:33:49.369850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.369811    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"]
Jul 18 01:33:49.370362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.370286    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:49.370362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.370299    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:49.370362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.370362    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5f8af0-c061-41df-839e-3074f069d1b5" containerName="revoke-keys"
Jul 18 01:33:49.370536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.370376    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3904a3a1-ddb4-43a5-a064-9015cd3e7b9c" containerName="revoke-keys"
Jul 18 01:33:49.374874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.374853    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:33:49.380292 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.380248    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"]
Jul 18 01:33:49.468510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.468466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ad1591-40ca-49c5-b336-01935e6b40d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh\" (UID: \"99ad1591-40ca-49c5-b336-01935e6b40d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:33:49.569597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.569564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ad1591-40ca-49c5-b336-01935e6b40d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh\" (UID: \"99ad1591-40ca-49c5-b336-01935e6b40d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:33:49.570185 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.570167    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ad1591-40ca-49c5-b336-01935e6b40d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh\" (UID: \"99ad1591-40ca-49c5-b336-01935e6b40d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:33:49.688709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.688617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:33:49.829583 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.829556    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"]
Jul 18 01:33:49.831413 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:33:49.831385    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ad1591_40ca_49c5_b336_01935e6b40d5.slice/crio-79dc9b273345df6f1fd6361903ad6b4a672020b4ebe84f39f7e774cf64de5d98 WatchSource:0}: Error finding container 79dc9b273345df6f1fd6361903ad6b4a672020b4ebe84f39f7e774cf64de5d98: Status 404 returned error can't find the container with id 79dc9b273345df6f1fd6361903ad6b4a672020b4ebe84f39f7e774cf64de5d98
Jul 18 01:33:49.903294 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:49.903249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" event={"ID":"99ad1591-40ca-49c5-b336-01935e6b40d5","Type":"ContainerStarted","Data":"79dc9b273345df6f1fd6361903ad6b4a672020b4ebe84f39f7e774cf64de5d98"}
Jul 18 01:33:50.910432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:50.910333    2567 generic.go:356] "Generic (PLEG): container finished" podID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerID="d0f64c4656930039cd2a6b0b71944c14166939d601419988b165d01df7a5aeaf" exitCode=6
Jul 18 01:33:50.910432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:50.910384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" event={"ID":"99ad1591-40ca-49c5-b336-01935e6b40d5","Type":"ContainerDied","Data":"d0f64c4656930039cd2a6b0b71944c14166939d601419988b165d01df7a5aeaf"}
Jul 18 01:33:50.910850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:50.910617    2567 scope.go:117] "RemoveContainer" containerID="d0f64c4656930039cd2a6b0b71944c14166939d601419988b165d01df7a5aeaf"
Jul 18 01:33:51.536228 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.536191    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"]
Jul 18 01:33:51.539715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.539698    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:33:51.545862 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.545836    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"]
Jul 18 01:33:51.690209 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.690166    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/489ef748-b106-4e13-bebe-9032997c66e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc\" (UID: \"489ef748-b106-4e13-bebe-9032997c66e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:33:51.790770 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.790668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/489ef748-b106-4e13-bebe-9032997c66e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc\" (UID: \"489ef748-b106-4e13-bebe-9032997c66e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:33:51.791284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.791248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/489ef748-b106-4e13-bebe-9032997c66e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc\" (UID: \"489ef748-b106-4e13-bebe-9032997c66e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:33:51.852990 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.852951    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:33:51.922455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.922418    2567 generic.go:356] "Generic (PLEG): container finished" podID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerID="fd2a318102ebb819e66e3553c016d556f92b86481f594cca720bc2d015d562b0" exitCode=6
Jul 18 01:33:51.924698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.923880    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" event={"ID":"99ad1591-40ca-49c5-b336-01935e6b40d5","Type":"ContainerDied","Data":"fd2a318102ebb819e66e3553c016d556f92b86481f594cca720bc2d015d562b0"}
Jul 18 01:33:51.924698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.923994    2567 scope.go:117] "RemoveContainer" containerID="d0f64c4656930039cd2a6b0b71944c14166939d601419988b165d01df7a5aeaf"
Jul 18 01:33:51.924698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:51.924111    2567 scope.go:117] "RemoveContainer" containerID="fd2a318102ebb819e66e3553c016d556f92b86481f594cca720bc2d015d562b0"
Jul 18 01:33:51.924698 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:51.924450    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh_odh-ai-gateway-infra(99ad1591-40ca-49c5-b336-01935e6b40d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5"
Jul 18 01:33:52.007121 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.007092    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"]
Jul 18 01:33:52.007759 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:33:52.007734    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489ef748_b106_4e13_bebe_9032997c66e8.slice/crio-4353c8ba5684c091964bfb6c9b7013a6ba72e68f2203d25da7d24ad9ba089e0f WatchSource:0}: Error finding container 4353c8ba5684c091964bfb6c9b7013a6ba72e68f2203d25da7d24ad9ba089e0f: Status 404 returned error can't find the container with id 4353c8ba5684c091964bfb6c9b7013a6ba72e68f2203d25da7d24ad9ba089e0f
Jul 18 01:33:52.913696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.913611    2567 scope.go:117] "RemoveContainer" containerID="747a4f3b6472c054c8af6cd04c4038abae0d6a89eef59def23ddebad27f9f277"
Jul 18 01:33:52.932780 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.932752    2567 scope.go:117] "RemoveContainer" containerID="fd2a318102ebb819e66e3553c016d556f92b86481f594cca720bc2d015d562b0"
Jul 18 01:33:52.933165 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:52.933018    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh_odh-ai-gateway-infra(99ad1591-40ca-49c5-b336-01935e6b40d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5"
Jul 18 01:33:52.934299 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.934247    2567 generic.go:356] "Generic (PLEG): container finished" podID="489ef748-b106-4e13-bebe-9032997c66e8" containerID="047533bfafb2eb413f63d85074fea33d35f8adb7bb16a1babda290b6d15dc897" exitCode=6
Jul 18 01:33:52.934436 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.934313    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" event={"ID":"489ef748-b106-4e13-bebe-9032997c66e8","Type":"ContainerDied","Data":"047533bfafb2eb413f63d85074fea33d35f8adb7bb16a1babda290b6d15dc897"}
Jul 18 01:33:52.934436 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.934372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" event={"ID":"489ef748-b106-4e13-bebe-9032997c66e8","Type":"ContainerStarted","Data":"4353c8ba5684c091964bfb6c9b7013a6ba72e68f2203d25da7d24ad9ba089e0f"}
Jul 18 01:33:52.934561 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:52.934547    2567 scope.go:117] "RemoveContainer" containerID="047533bfafb2eb413f63d85074fea33d35f8adb7bb16a1babda290b6d15dc897"
Jul 18 01:33:53.941905 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.941820    2567 generic.go:356] "Generic (PLEG): container finished" podID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerID="377bbe6cb06153d1889283cd2f6af9d895028ea6e511a07f03a1eeee22e1a69e" exitCode=6
Jul 18 01:33:53.942377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.941894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" event={"ID":"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30","Type":"ContainerDied","Data":"377bbe6cb06153d1889283cd2f6af9d895028ea6e511a07f03a1eeee22e1a69e"}
Jul 18 01:33:53.942377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.941952    2567 scope.go:117] "RemoveContainer" containerID="747a4f3b6472c054c8af6cd04c4038abae0d6a89eef59def23ddebad27f9f277"
Jul 18 01:33:53.942377 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.942233    2567 scope.go:117] "RemoveContainer" containerID="377bbe6cb06153d1889283cd2f6af9d895028ea6e511a07f03a1eeee22e1a69e"
Jul 18 01:33:53.948317 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:53.943004    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg_odh-ai-gateway-infra(27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30"
Jul 18 01:33:53.951655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.951624    2567 generic.go:356] "Generic (PLEG): container finished" podID="489ef748-b106-4e13-bebe-9032997c66e8" containerID="9ce947756622178d19db149eefd31b1fba366881490b22212021f6c3c94ef3f0" exitCode=6
Jul 18 01:33:53.951801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.951715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" event={"ID":"489ef748-b106-4e13-bebe-9032997c66e8","Type":"ContainerDied","Data":"9ce947756622178d19db149eefd31b1fba366881490b22212021f6c3c94ef3f0"}
Jul 18 01:33:53.951996 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.951982    2567 scope.go:117] "RemoveContainer" containerID="9ce947756622178d19db149eefd31b1fba366881490b22212021f6c3c94ef3f0"
Jul 18 01:33:53.952186 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:53.952165    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc_odh-ai-gateway-infra(489ef748-b106-4e13-bebe-9032997c66e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" podUID="489ef748-b106-4e13-bebe-9032997c66e8"
Jul 18 01:33:53.962311 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:53.962289    2567 scope.go:117] "RemoveContainer" containerID="047533bfafb2eb413f63d85074fea33d35f8adb7bb16a1babda290b6d15dc897"
Jul 18 01:33:54.959595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:54.959569    2567 scope.go:117] "RemoveContainer" containerID="9ce947756622178d19db149eefd31b1fba366881490b22212021f6c3c94ef3f0"
Jul 18 01:33:54.960094 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:33:54.959820    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc_odh-ai-gateway-infra(489ef748-b106-4e13-bebe-9032997c66e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" podUID="489ef748-b106-4e13-bebe-9032997c66e8"
Jul 18 01:33:54.966763 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:54.966733    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"]
Jul 18 01:33:55.113114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.113087    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:55.123666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.123639    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30-maas-api-service-ca\") pod \"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30\" (UID: \"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30\") "
Jul 18 01:33:55.123966 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.123945    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" (UID: "27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:33:55.224914 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.224828    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:33:55.967736 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.967704    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"
Jul 18 01:33:55.968291 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.967713    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg" event={"ID":"27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30","Type":"ContainerDied","Data":"12b679f3416e4accaef22a81bfe33fa19d6a41f99b354e95e0c50ebf352ebee8"}
Jul 18 01:33:55.968291 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.967830    2567 scope.go:117] "RemoveContainer" containerID="377bbe6cb06153d1889283cd2f6af9d895028ea6e511a07f03a1eeee22e1a69e"
Jul 18 01:33:55.988685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.988657    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"]
Jul 18 01:33:55.992524 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:55.992500    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-zmbqg"]
Jul 18 01:33:56.919722 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:33:56.919690    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" path="/var/lib/kubelet/pods/27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30/volumes"
Jul 18 01:34:01.911806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:01.911774    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:34:06.598078 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598036    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"]
Jul 18 01:34:06.598657 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598635    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:06.598744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598659    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:06.598744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598679    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:06.598744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598687    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:06.598908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598809    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:06.598908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.598824    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:06.606909 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.602887    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:06.609835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.609807    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"]
Jul 18 01:34:06.620931 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.620906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cf9b-c425-4201-926d-1419af146f50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-c2574\" (UID: \"a972cf9b-c425-4201-926d-1419af146f50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:06.721857 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.721815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cf9b-c425-4201-926d-1419af146f50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-c2574\" (UID: \"a972cf9b-c425-4201-926d-1419af146f50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:06.722442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.722424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cf9b-c425-4201-926d-1419af146f50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-c2574\" (UID: \"a972cf9b-c425-4201-926d-1419af146f50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:06.914122 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.914040    2567 scope.go:117] "RemoveContainer" containerID="fd2a318102ebb819e66e3553c016d556f92b86481f594cca720bc2d015d562b0"
Jul 18 01:34:06.919035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:06.919014    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:07.066833 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:07.066804    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"]
Jul 18 01:34:07.068071 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:34:07.068036    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda972cf9b_c425_4201_926d_1419af146f50.slice/crio-58a3c3ad9da5e309ef8a02dbb1e9ed18c9ef19d36493486f9686a484284cc249 WatchSource:0}: Error finding container 58a3c3ad9da5e309ef8a02dbb1e9ed18c9ef19d36493486f9686a484284cc249: Status 404 returned error can't find the container with id 58a3c3ad9da5e309ef8a02dbb1e9ed18c9ef19d36493486f9686a484284cc249
Jul 18 01:34:08.039658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.039624    2567 generic.go:356] "Generic (PLEG): container finished" podID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerID="06d35caff03db51f4d720d4848c35a34e1223b19f012602b0eee75b5c5f15744" exitCode=6
Jul 18 01:34:08.040107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.039697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" event={"ID":"99ad1591-40ca-49c5-b336-01935e6b40d5","Type":"ContainerDied","Data":"06d35caff03db51f4d720d4848c35a34e1223b19f012602b0eee75b5c5f15744"}
Jul 18 01:34:08.040107 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.039764    2567 scope.go:117] "RemoveContainer" containerID="fd2a318102ebb819e66e3553c016d556f92b86481f594cca720bc2d015d562b0"
Jul 18 01:34:08.040240 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.040108    2567 scope.go:117] "RemoveContainer" containerID="06d35caff03db51f4d720d4848c35a34e1223b19f012602b0eee75b5c5f15744"
Jul 18 01:34:08.040381 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:08.040352    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh_odh-ai-gateway-infra(99ad1591-40ca-49c5-b336-01935e6b40d5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5"
Jul 18 01:34:08.041589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.041570    2567 generic.go:356] "Generic (PLEG): container finished" podID="a972cf9b-c425-4201-926d-1419af146f50" containerID="ae9327ee89a5a4c906a3c15ddee2d99535b5bc6719ed4be63c081c773c6b8291" exitCode=6
Jul 18 01:34:08.041708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.041646    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" event={"ID":"a972cf9b-c425-4201-926d-1419af146f50","Type":"ContainerDied","Data":"ae9327ee89a5a4c906a3c15ddee2d99535b5bc6719ed4be63c081c773c6b8291"}
Jul 18 01:34:08.041708 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.041692    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" event={"ID":"a972cf9b-c425-4201-926d-1419af146f50","Type":"ContainerStarted","Data":"58a3c3ad9da5e309ef8a02dbb1e9ed18c9ef19d36493486f9686a484284cc249"}
Jul 18 01:34:08.041868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.041852    2567 scope.go:117] "RemoveContainer" containerID="ae9327ee89a5a4c906a3c15ddee2d99535b5bc6719ed4be63c081c773c6b8291"
Jul 18 01:34:08.913651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:08.913611    2567 scope.go:117] "RemoveContainer" containerID="9ce947756622178d19db149eefd31b1fba366881490b22212021f6c3c94ef3f0"
Jul 18 01:34:09.050589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.050507    2567 generic.go:356] "Generic (PLEG): container finished" podID="a972cf9b-c425-4201-926d-1419af146f50" containerID="583cee7328456341c9da46a44bf86b84907d4f6ae61aa7fcd663e99c0f2f6d5d" exitCode=6
Jul 18 01:34:09.050589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.050553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" event={"ID":"a972cf9b-c425-4201-926d-1419af146f50","Type":"ContainerDied","Data":"583cee7328456341c9da46a44bf86b84907d4f6ae61aa7fcd663e99c0f2f6d5d"}
Jul 18 01:34:09.051133 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.050608    2567 scope.go:117] "RemoveContainer" containerID="ae9327ee89a5a4c906a3c15ddee2d99535b5bc6719ed4be63c081c773c6b8291"
Jul 18 01:34:09.051133 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.050841    2567 scope.go:117] "RemoveContainer" containerID="583cee7328456341c9da46a44bf86b84907d4f6ae61aa7fcd663e99c0f2f6d5d"
Jul 18 01:34:09.051133 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:09.051117    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-c2574_odh-ai-gateway-infra(a972cf9b-c425-4201-926d-1419af146f50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" podUID="a972cf9b-c425-4201-926d-1419af146f50"
Jul 18 01:34:09.063247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.063219    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"]
Jul 18 01:34:09.276483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.276451    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:34:09.344054 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.344019    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ad1591-40ca-49c5-b336-01935e6b40d5-maas-api-service-ca\") pod \"99ad1591-40ca-49c5-b336-01935e6b40d5\" (UID: \"99ad1591-40ca-49c5-b336-01935e6b40d5\") "
Jul 18 01:34:09.344424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.344399    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99ad1591-40ca-49c5-b336-01935e6b40d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99ad1591-40ca-49c5-b336-01935e6b40d5" (UID: "99ad1591-40ca-49c5-b336-01935e6b40d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:34:09.445763 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:09.445729    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ad1591-40ca-49c5-b336-01935e6b40d5-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:34:10.058733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.058705    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"
Jul 18 01:34:10.059175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.058734    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh" event={"ID":"99ad1591-40ca-49c5-b336-01935e6b40d5","Type":"ContainerDied","Data":"79dc9b273345df6f1fd6361903ad6b4a672020b4ebe84f39f7e774cf64de5d98"}
Jul 18 01:34:10.059175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.058809    2567 scope.go:117] "RemoveContainer" containerID="06d35caff03db51f4d720d4848c35a34e1223b19f012602b0eee75b5c5f15744"
Jul 18 01:34:10.061140 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.061119    2567 scope.go:117] "RemoveContainer" containerID="583cee7328456341c9da46a44bf86b84907d4f6ae61aa7fcd663e99c0f2f6d5d"
Jul 18 01:34:10.061454 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:10.061430    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-c2574_odh-ai-gateway-infra(a972cf9b-c425-4201-926d-1419af146f50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" podUID="a972cf9b-c425-4201-926d-1419af146f50"
Jul 18 01:34:10.063219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.063195    2567 generic.go:356] "Generic (PLEG): container finished" podID="489ef748-b106-4e13-bebe-9032997c66e8" containerID="d5368b1a4e2ea8a609abe336a6902f3adc1439f3246ed58ad119461ce63c2b5d" exitCode=6
Jul 18 01:34:10.063360 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.063297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" event={"ID":"489ef748-b106-4e13-bebe-9032997c66e8","Type":"ContainerDied","Data":"d5368b1a4e2ea8a609abe336a6902f3adc1439f3246ed58ad119461ce63c2b5d"}
Jul 18 01:34:10.063624 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.063607    2567 scope.go:117] "RemoveContainer" containerID="d5368b1a4e2ea8a609abe336a6902f3adc1439f3246ed58ad119461ce63c2b5d"
Jul 18 01:34:10.063828 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:10.063807    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc_odh-ai-gateway-infra(489ef748-b106-4e13-bebe-9032997c66e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" podUID="489ef748-b106-4e13-bebe-9032997c66e8"
Jul 18 01:34:10.072446 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.072427    2567 scope.go:117] "RemoveContainer" containerID="9ce947756622178d19db149eefd31b1fba366881490b22212021f6c3c94ef3f0"
Jul 18 01:34:10.090479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.090449    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"]
Jul 18 01:34:10.093971 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.093941    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-g8bnh"]
Jul 18 01:34:10.919538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:10.919502    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" path="/var/lib/kubelet/pods/99ad1591-40ca-49c5-b336-01935e6b40d5/volumes"
Jul 18 01:34:11.115234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:11.115146    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"]
Jul 18 01:34:11.259969 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:11.259936    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:34:11.364618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:11.364576    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/489ef748-b106-4e13-bebe-9032997c66e8-maas-api-service-ca\") pod \"489ef748-b106-4e13-bebe-9032997c66e8\" (UID: \"489ef748-b106-4e13-bebe-9032997c66e8\") "
Jul 18 01:34:11.364955 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:11.364931    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/489ef748-b106-4e13-bebe-9032997c66e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "489ef748-b106-4e13-bebe-9032997c66e8" (UID: "489ef748-b106-4e13-bebe-9032997c66e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:34:11.466303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:11.466204    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/489ef748-b106-4e13-bebe-9032997c66e8-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:34:12.079660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:12.079629    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"
Jul 18 01:34:12.079972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:12.079626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc" event={"ID":"489ef748-b106-4e13-bebe-9032997c66e8","Type":"ContainerDied","Data":"4353c8ba5684c091964bfb6c9b7013a6ba72e68f2203d25da7d24ad9ba089e0f"}
Jul 18 01:34:12.079972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:12.079780    2567 scope.go:117] "RemoveContainer" containerID="d5368b1a4e2ea8a609abe336a6902f3adc1439f3246ed58ad119461ce63c2b5d"
Jul 18 01:34:12.099858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:12.099831    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"]
Jul 18 01:34:12.103507 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:12.103485    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-td6wc"]
Jul 18 01:34:12.920058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:12.920024    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="489ef748-b106-4e13-bebe-9032997c66e8" path="/var/lib/kubelet/pods/489ef748-b106-4e13-bebe-9032997c66e8/volumes"
Jul 18 01:34:19.506738 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:19.506700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:34:21.691517 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:21.691484    2567 scope.go:117] "RemoveContainer" containerID="332eb553438c9ca2a18f7d4df415ac8eae6d7b488c42894bc133d45031bccd8a"
Jul 18 01:34:22.913846 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:22.913804    2567 scope.go:117] "RemoveContainer" containerID="583cee7328456341c9da46a44bf86b84907d4f6ae61aa7fcd663e99c0f2f6d5d"
Jul 18 01:34:24.148408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:24.148372    2567 generic.go:356] "Generic (PLEG): container finished" podID="a972cf9b-c425-4201-926d-1419af146f50" containerID="ee5fcb83cf6fb529bbec9d44ad549a9fe8876bd48b1a9649260457bf54fddf2a" exitCode=6
Jul 18 01:34:24.148847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:24.148447    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" event={"ID":"a972cf9b-c425-4201-926d-1419af146f50","Type":"ContainerDied","Data":"ee5fcb83cf6fb529bbec9d44ad549a9fe8876bd48b1a9649260457bf54fddf2a"}
Jul 18 01:34:24.148847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:24.148508    2567 scope.go:117] "RemoveContainer" containerID="583cee7328456341c9da46a44bf86b84907d4f6ae61aa7fcd663e99c0f2f6d5d"
Jul 18 01:34:24.148847 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:24.148779    2567 scope.go:117] "RemoveContainer" containerID="ee5fcb83cf6fb529bbec9d44ad549a9fe8876bd48b1a9649260457bf54fddf2a"
Jul 18 01:34:24.149034 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:24.149014    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-c2574_odh-ai-gateway-infra(a972cf9b-c425-4201-926d-1419af146f50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" podUID="a972cf9b-c425-4201-926d-1419af146f50"
Jul 18 01:34:25.173363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:25.173330    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"]
Jul 18 01:34:25.314019 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:25.313988    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:25.392014 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:25.391971    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cf9b-c425-4201-926d-1419af146f50-maas-api-service-ca\") pod \"a972cf9b-c425-4201-926d-1419af146f50\" (UID: \"a972cf9b-c425-4201-926d-1419af146f50\") "
Jul 18 01:34:25.392396 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:25.392373    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a972cf9b-c425-4201-926d-1419af146f50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a972cf9b-c425-4201-926d-1419af146f50" (UID: "a972cf9b-c425-4201-926d-1419af146f50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:34:25.493384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:25.493262    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cf9b-c425-4201-926d-1419af146f50-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:34:26.163626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:26.163590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574" event={"ID":"a972cf9b-c425-4201-926d-1419af146f50","Type":"ContainerDied","Data":"58a3c3ad9da5e309ef8a02dbb1e9ed18c9ef19d36493486f9686a484284cc249"}
Jul 18 01:34:26.163626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:26.163612    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"
Jul 18 01:34:26.163888 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:26.163652    2567 scope.go:117] "RemoveContainer" containerID="ee5fcb83cf6fb529bbec9d44ad549a9fe8876bd48b1a9649260457bf54fddf2a"
Jul 18 01:34:26.186228 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:26.186197    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"]
Jul 18 01:34:26.192756 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:26.192727    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-c2574"]
Jul 18 01:34:26.919501 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:26.919467    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a972cf9b-c425-4201-926d-1419af146f50" path="/var/lib/kubelet/pods/a972cf9b-c425-4201-926d-1419af146f50/volumes"
Jul 18 01:34:34.011139 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011097    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"]
Jul 18 01:34:34.011691 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011670    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011695    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011714    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011723    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011732    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011741    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011758    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011767    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011778    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011786    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011805    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011813    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011826    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.011839 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011834    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011846    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011854    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011959    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27687bf5-c4ec-45e8-aba9-cf3d9ac0ab30" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011970    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011978    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011989    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.011999    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.012008    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.012017    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:34.012528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.012029    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:34.015386 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.015364    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:34.020097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.020074    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:34:34.021573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.021549    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:34:34.037389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.037364    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"]
Jul 18 01:34:34.178033 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.177994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534ae410-6b97-41d6-8b9b-418ae8b61a3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj\" (UID: \"534ae410-6b97-41d6-8b9b-418ae8b61a3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:34.279170 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.279078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534ae410-6b97-41d6-8b9b-418ae8b61a3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj\" (UID: \"534ae410-6b97-41d6-8b9b-418ae8b61a3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:34.279721 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.279700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534ae410-6b97-41d6-8b9b-418ae8b61a3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj\" (UID: \"534ae410-6b97-41d6-8b9b-418ae8b61a3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:34.325697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.325662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:34.472255 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.472227    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"]
Jul 18 01:34:34.474078 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:34:34.474043    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534ae410_6b97_41d6_8b9b_418ae8b61a3b.slice/crio-e279a0b408bd3e67dc8ce2a85b9afdaeb9e0a32596f15f7f78e813835599430a WatchSource:0}: Error finding container e279a0b408bd3e67dc8ce2a85b9afdaeb9e0a32596f15f7f78e813835599430a: Status 404 returned error can't find the container with id e279a0b408bd3e67dc8ce2a85b9afdaeb9e0a32596f15f7f78e813835599430a
Jul 18 01:34:34.859169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:34.859133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:34:35.216104 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:35.216069    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerStarted","Data":"335e2d5bae0685045a9642bd62b9fc77ec53af41a8f0561f4df30c7403c33cfb"}
Jul 18 01:34:35.216538 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:35.216127    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerStarted","Data":"e279a0b408bd3e67dc8ce2a85b9afdaeb9e0a32596f15f7f78e813835599430a"}
Jul 18 01:34:35.229998 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:35.229948    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" podStartSLOduration=2.229930287 podStartE2EDuration="2.229930287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:34:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:34:35.22847178 +0000 UTC m=+3254.755976739" watchObservedRunningTime="2026-07-18 01:34:35.229930287 +0000 UTC m=+3254.757435256"
Jul 18 01:34:36.055450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:36.055414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:34:36.223528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:36.223500    2567 generic.go:356] "Generic (PLEG): container finished" podID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerID="335e2d5bae0685045a9642bd62b9fc77ec53af41a8f0561f4df30c7403c33cfb" exitCode=6
Jul 18 01:34:36.223932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:36.223590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerDied","Data":"335e2d5bae0685045a9642bd62b9fc77ec53af41a8f0561f4df30c7403c33cfb"}
Jul 18 01:34:36.223932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:36.223754    2567 scope.go:117] "RemoveContainer" containerID="335e2d5bae0685045a9642bd62b9fc77ec53af41a8f0561f4df30c7403c33cfb"
Jul 18 01:34:37.230633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:37.230542    2567 generic.go:356] "Generic (PLEG): container finished" podID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerID="d8e50395d6356f1d11cd48d1d28eb24a3ac53c0e420a03d9126cb4bbe113a837" exitCode=6
Jul 18 01:34:37.230633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:37.230603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerDied","Data":"d8e50395d6356f1d11cd48d1d28eb24a3ac53c0e420a03d9126cb4bbe113a837"}
Jul 18 01:34:37.231129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:37.230651    2567 scope.go:117] "RemoveContainer" containerID="335e2d5bae0685045a9642bd62b9fc77ec53af41a8f0561f4df30c7403c33cfb"
Jul 18 01:34:37.231129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:37.230807    2567 scope.go:117] "RemoveContainer" containerID="d8e50395d6356f1d11cd48d1d28eb24a3ac53c0e420a03d9126cb4bbe113a837"
Jul 18 01:34:37.231129 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:37.231077    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj_odh-ai-gateway-infra(534ae410-6b97-41d6-8b9b-418ae8b61a3b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b"
Jul 18 01:34:37.904156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:37.904114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:34:38.237911 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:38.237824    2567 scope.go:117] "RemoveContainer" containerID="d8e50395d6356f1d11cd48d1d28eb24a3ac53c0e420a03d9126cb4bbe113a837"
Jul 18 01:34:38.238333 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:38.238029    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj_odh-ai-gateway-infra(534ae410-6b97-41d6-8b9b-418ae8b61a3b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b"
Jul 18 01:34:39.411993 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.411954    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"]
Jul 18 01:34:39.412466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.412391    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:39.412466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.412404    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:39.412466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.412412    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:39.412466 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.412418    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a972cf9b-c425-4201-926d-1419af146f50" containerName="revoke-keys"
Jul 18 01:34:39.412696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.412496    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ad1591-40ca-49c5-b336-01935e6b40d5" containerName="revoke-keys"
Jul 18 01:34:39.412696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.412509    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="489ef748-b106-4e13-bebe-9032997c66e8" containerName="revoke-keys"
Jul 18 01:34:39.415635 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.415614    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.418058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.418031    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6b3233\"/\"kube-root-ca.crt\""
Jul 18 01:34:39.418201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.418066    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6b3233\"/\"openshift-service-ca.crt\""
Jul 18 01:34:39.418901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.418885    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b3233\"/\"default-dockercfg-wj657\""
Jul 18 01:34:39.419051 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.418904    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b3233\"/\"sub-test-model-6b3233-kserve-self-signed-certs\""
Jul 18 01:34:39.425753 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.425730    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"]
Jul 18 01:34:39.447831 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.447793    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:34:39.530342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.530311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-home\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.530342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.530347    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tmp-dir\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.530558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.530370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-dshm\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.530558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.530445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p22cz\" (UniqueName: \"kubernetes.io/projected/90b1afdb-a84d-4f33-b207-ed4731d9c12c-kube-api-access-p22cz\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.530558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.530527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-model-cache\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.530558 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.530543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tls-certs\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-home\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631151 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tmp-dir\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-dshm\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p22cz\" (UniqueName: \"kubernetes.io/projected/90b1afdb-a84d-4f33-b207-ed4731d9c12c-kube-api-access-p22cz\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-model-cache\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tls-certs\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-home\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tmp-dir\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.631786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.631702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-model-cache\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.633765 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.633747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-dshm\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.633942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.633925    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tls-certs\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.638742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.638720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p22cz\" (UniqueName: \"kubernetes.io/projected/90b1afdb-a84d-4f33-b207-ed4731d9c12c-kube-api-access-p22cz\") pod \"sub-test-model-6b3233-kserve-86564958bf-7zqcs\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") " pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.725641 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.725546    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:39.866575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:39.866548    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"]
Jul 18 01:34:39.867891 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:34:39.867862    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90b1afdb_a84d_4f33_b207_ed4731d9c12c.slice/crio-8ef38e591ae9b6d7dcfaa04a5c6f618563510395926ab130a469e8928a4102b3 WatchSource:0}: Error finding container 8ef38e591ae9b6d7dcfaa04a5c6f618563510395926ab130a469e8928a4102b3: Status 404 returned error can't find the container with id 8ef38e591ae9b6d7dcfaa04a5c6f618563510395926ab130a469e8928a4102b3
Jul 18 01:34:40.251866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:40.251820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs" event={"ID":"90b1afdb-a84d-4f33-b207-ed4731d9c12c","Type":"ContainerStarted","Data":"8ef38e591ae9b6d7dcfaa04a5c6f618563510395926ab130a469e8928a4102b3"}
Jul 18 01:34:41.260660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:41.260565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs" event={"ID":"90b1afdb-a84d-4f33-b207-ed4731d9c12c","Type":"ContainerStarted","Data":"f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380"}
Jul 18 01:34:41.278841 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:41.278786    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs" podStartSLOduration=1.257499992 podStartE2EDuration="2.278771319s" totalImagesPullingTime="1.021271327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:34:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:34:41.276743753 +0000 UTC m=+3260.804248713" watchObservedRunningTime="2026-07-18 01:34:41.278771319 +0000 UTC m=+3260.806276278"
Jul 18 01:34:47.156237 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:47.156198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:34:49.408631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.408595    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"]
Jul 18 01:34:49.412378 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.412359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:34:49.419243 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.419214    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"]
Jul 18 01:34:49.517668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.517629    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-p652d\" (UID: \"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:34:49.618530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.618489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-p652d\" (UID: \"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:34:49.619129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.619107    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-p652d\" (UID: \"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:34:49.725588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.725499    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:34:49.726610 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.726583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:49.726761 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.726677    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:49.742781 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:49.742757    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:50.084633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:50.084609    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"]
Jul 18 01:34:50.086823 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:34:50.086792    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c090e2_209d_4a5d_8ae6_2a64ecbd09ed.slice/crio-2819242b914e5f2ab2f8766ea10a21dbeb1219e3ea549b41d5110bb23f7026d8 WatchSource:0}: Error finding container 2819242b914e5f2ab2f8766ea10a21dbeb1219e3ea549b41d5110bb23f7026d8: Status 404 returned error can't find the container with id 2819242b914e5f2ab2f8766ea10a21dbeb1219e3ea549b41d5110bb23f7026d8
Jul 18 01:34:50.313393 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:50.313361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerStarted","Data":"e4e1f1c403ef1487722b22dfb4f8bdde6bb76bd896e3ad74c8ba2c2ee178344f"}
Jul 18 01:34:50.313533 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:50.313414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerStarted","Data":"2819242b914e5f2ab2f8766ea10a21dbeb1219e3ea549b41d5110bb23f7026d8"}
Jul 18 01:34:50.325181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:50.325152    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:34:50.327166 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:50.327098    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" podStartSLOduration=1.327083446 podStartE2EDuration="1.327083446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:34:50.325946914 +0000 UTC m=+3269.853451874" watchObservedRunningTime="2026-07-18 01:34:50.327083446 +0000 UTC m=+3269.854588405"
Jul 18 01:34:51.319966 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.319933    2567 generic.go:356] "Generic (PLEG): container finished" podID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerID="e4e1f1c403ef1487722b22dfb4f8bdde6bb76bd896e3ad74c8ba2c2ee178344f" exitCode=6
Jul 18 01:34:51.320389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.320017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerDied","Data":"e4e1f1c403ef1487722b22dfb4f8bdde6bb76bd896e3ad74c8ba2c2ee178344f"}
Jul 18 01:34:51.320389 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.320244    2567 scope.go:117] "RemoveContainer" containerID="e4e1f1c403ef1487722b22dfb4f8bdde6bb76bd896e3ad74c8ba2c2ee178344f"
Jul 18 01:34:51.577886 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.577857    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"]
Jul 18 01:34:51.582747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.582723    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:34:51.588578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.588547    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"]
Jul 18 01:34:51.637579 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.637538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3d52bb-b651-4b68-a671-1433c4277552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq\" (UID: \"2a3d52bb-b651-4b68-a671-1433c4277552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:34:51.738100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.738054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3d52bb-b651-4b68-a671-1433c4277552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq\" (UID: \"2a3d52bb-b651-4b68-a671-1433c4277552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:34:51.738732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.738712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3d52bb-b651-4b68-a671-1433c4277552-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq\" (UID: \"2a3d52bb-b651-4b68-a671-1433c4277552\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:34:51.896141 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:51.896059    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:34:52.042445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.042420    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"]
Jul 18 01:34:52.045423 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:34:52.045388    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3d52bb_b651_4b68_a671_1433c4277552.slice/crio-801ba5e8c999eafeee4e440c4c3871c3c8e50bfc722eec38c3b6dc54af3babc4 WatchSource:0}: Error finding container 801ba5e8c999eafeee4e440c4c3871c3c8e50bfc722eec38c3b6dc54af3babc4: Status 404 returned error can't find the container with id 801ba5e8c999eafeee4e440c4c3871c3c8e50bfc722eec38c3b6dc54af3babc4
Jul 18 01:34:52.328289 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.328227    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" event={"ID":"2a3d52bb-b651-4b68-a671-1433c4277552","Type":"ContainerStarted","Data":"801ba5e8c999eafeee4e440c4c3871c3c8e50bfc722eec38c3b6dc54af3babc4"}
Jul 18 01:34:52.330195 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.330170    2567 generic.go:356] "Generic (PLEG): container finished" podID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerID="c8ea57cd621fb92545683404952646eff6c7f1c353e27f2413061285e0119f70" exitCode=6
Jul 18 01:34:52.330363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.330227    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerDied","Data":"c8ea57cd621fb92545683404952646eff6c7f1c353e27f2413061285e0119f70"}
Jul 18 01:34:52.330363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.330295    2567 scope.go:117] "RemoveContainer" containerID="e4e1f1c403ef1487722b22dfb4f8bdde6bb76bd896e3ad74c8ba2c2ee178344f"
Jul 18 01:34:52.330518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.330503    2567 scope.go:117] "RemoveContainer" containerID="c8ea57cd621fb92545683404952646eff6c7f1c353e27f2413061285e0119f70"
Jul 18 01:34:52.330791 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:52.330751    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-p652d_odh-ai-gateway-infra(28c090e2-209d-4a5d-8ae6-2a64ecbd09ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed"
Jul 18 01:34:52.913494 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:52.913469    2567 scope.go:117] "RemoveContainer" containerID="d8e50395d6356f1d11cd48d1d28eb24a3ac53c0e420a03d9126cb4bbe113a837"
Jul 18 01:34:53.337442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:53.337351    2567 generic.go:356] "Generic (PLEG): container finished" podID="2a3d52bb-b651-4b68-a671-1433c4277552" containerID="48dd3e7c0314bd03b93ecb0ccb7da952dc08fc9b2c5edd7aabd3423863b66a66" exitCode=6
Jul 18 01:34:53.337442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:53.337414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" event={"ID":"2a3d52bb-b651-4b68-a671-1433c4277552","Type":"ContainerDied","Data":"48dd3e7c0314bd03b93ecb0ccb7da952dc08fc9b2c5edd7aabd3423863b66a66"}
Jul 18 01:34:53.337903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:53.337648    2567 scope.go:117] "RemoveContainer" containerID="48dd3e7c0314bd03b93ecb0ccb7da952dc08fc9b2c5edd7aabd3423863b66a66"
Jul 18 01:34:53.339486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:53.339469    2567 scope.go:117] "RemoveContainer" containerID="c8ea57cd621fb92545683404952646eff6c7f1c353e27f2413061285e0119f70"
Jul 18 01:34:53.339706 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:53.339682    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-p652d_odh-ai-gateway-infra(28c090e2-209d-4a5d-8ae6-2a64ecbd09ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed"
Jul 18 01:34:53.341218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:53.341194    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerStarted","Data":"21e4eccc4184bb1acee21d90c109d6100dd27ad1c946895eebe27ddf1b480161"}
Jul 18 01:34:54.065945 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.065914    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:34:54.126901 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.126874    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:34:54.348863 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.348827    2567 generic.go:356] "Generic (PLEG): container finished" podID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerID="21e4eccc4184bb1acee21d90c109d6100dd27ad1c946895eebe27ddf1b480161" exitCode=6
Jul 18 01:34:54.349381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.348895    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerDied","Data":"21e4eccc4184bb1acee21d90c109d6100dd27ad1c946895eebe27ddf1b480161"}
Jul 18 01:34:54.349381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.348952    2567 scope.go:117] "RemoveContainer" containerID="d8e50395d6356f1d11cd48d1d28eb24a3ac53c0e420a03d9126cb4bbe113a837"
Jul 18 01:34:54.349381 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.349294    2567 scope.go:117] "RemoveContainer" containerID="21e4eccc4184bb1acee21d90c109d6100dd27ad1c946895eebe27ddf1b480161"
Jul 18 01:34:54.349596 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:54.349574    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj_odh-ai-gateway-infra(534ae410-6b97-41d6-8b9b-418ae8b61a3b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b"
Jul 18 01:34:54.351043 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.351018    2567 generic.go:356] "Generic (PLEG): container finished" podID="2a3d52bb-b651-4b68-a671-1433c4277552" containerID="1e20ecd1ab5694370775192a0a69eef2d598754dc31f7ff5785d0d99e55afacb" exitCode=6
Jul 18 01:34:54.351162 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.351070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" event={"ID":"2a3d52bb-b651-4b68-a671-1433c4277552","Type":"ContainerDied","Data":"1e20ecd1ab5694370775192a0a69eef2d598754dc31f7ff5785d0d99e55afacb"}
Jul 18 01:34:54.351325 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.351307    2567 scope.go:117] "RemoveContainer" containerID="1e20ecd1ab5694370775192a0a69eef2d598754dc31f7ff5785d0d99e55afacb"
Jul 18 01:34:54.351531 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:54.351511    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq_odh-ai-gateway-infra(2a3d52bb-b651-4b68-a671-1433c4277552)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" podUID="2a3d52bb-b651-4b68-a671-1433c4277552"
Jul 18 01:34:54.362057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.362030    2567 scope.go:117] "RemoveContainer" containerID="48dd3e7c0314bd03b93ecb0ccb7da952dc08fc9b2c5edd7aabd3423863b66a66"
Jul 18 01:34:54.376457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:54.376434    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"]
Jul 18 01:34:55.361487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:55.361462    2567 scope.go:117] "RemoveContainer" containerID="1e20ecd1ab5694370775192a0a69eef2d598754dc31f7ff5785d0d99e55afacb"
Jul 18 01:34:55.361832 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:34:55.361741    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq_odh-ai-gateway-infra(2a3d52bb-b651-4b68-a671-1433c4277552)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" podUID="2a3d52bb-b651-4b68-a671-1433c4277552"
Jul 18 01:34:55.499104 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:55.499074    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:55.578005 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:55.577966    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534ae410-6b97-41d6-8b9b-418ae8b61a3b-maas-api-service-ca\") pod \"534ae410-6b97-41d6-8b9b-418ae8b61a3b\" (UID: \"534ae410-6b97-41d6-8b9b-418ae8b61a3b\") "
Jul 18 01:34:55.578359 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:55.578338    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/534ae410-6b97-41d6-8b9b-418ae8b61a3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "534ae410-6b97-41d6-8b9b-418ae8b61a3b" (UID: "534ae410-6b97-41d6-8b9b-418ae8b61a3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:34:55.679002 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:55.678910    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534ae410-6b97-41d6-8b9b-418ae8b61a3b-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:34:56.100655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.100618    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:34:56.369405 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.369308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj" event={"ID":"534ae410-6b97-41d6-8b9b-418ae8b61a3b","Type":"ContainerDied","Data":"e279a0b408bd3e67dc8ce2a85b9afdaeb9e0a32596f15f7f78e813835599430a"}
Jul 18 01:34:56.369405 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.369343    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"
Jul 18 01:34:56.369405 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.369397    2567 scope.go:117] "RemoveContainer" containerID="21e4eccc4184bb1acee21d90c109d6100dd27ad1c946895eebe27ddf1b480161"
Jul 18 01:34:56.389887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.389857    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"]
Jul 18 01:34:56.395785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.395760    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-v8mlj"]
Jul 18 01:34:56.920178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:34:56.920139    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" path="/var/lib/kubelet/pods/534ae410-6b97-41d6-8b9b-418ae8b61a3b/volumes"
Jul 18 01:35:05.913717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:05.913687    2567 scope.go:117] "RemoveContainer" containerID="c8ea57cd621fb92545683404952646eff6c7f1c353e27f2413061285e0119f70"
Jul 18 01:35:06.443144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:06.443106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerStarted","Data":"0e020781fb69c70f93cdd1df324077eb372b7ac396d15d19e0ef67a831fd136b"}
Jul 18 01:35:07.450219 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:07.450187    2567 generic.go:356] "Generic (PLEG): container finished" podID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerID="0e020781fb69c70f93cdd1df324077eb372b7ac396d15d19e0ef67a831fd136b" exitCode=6
Jul 18 01:35:07.450683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:07.450235    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerDied","Data":"0e020781fb69c70f93cdd1df324077eb372b7ac396d15d19e0ef67a831fd136b"}
Jul 18 01:35:07.450683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:07.450295    2567 scope.go:117] "RemoveContainer" containerID="c8ea57cd621fb92545683404952646eff6c7f1c353e27f2413061285e0119f70"
Jul 18 01:35:07.450683 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:07.450529    2567 scope.go:117] "RemoveContainer" containerID="0e020781fb69c70f93cdd1df324077eb372b7ac396d15d19e0ef67a831fd136b"
Jul 18 01:35:07.450889 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:07.450798    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-p652d_odh-ai-gateway-infra(28c090e2-209d-4a5d-8ae6-2a64ecbd09ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed"
Jul 18 01:35:07.467526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:07.467502    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"]
Jul 18 01:35:08.599590 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:08.599561    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:35:08.704655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:08.704620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed-maas-api-service-ca\") pod \"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed\" (UID: \"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed\") "
Jul 18 01:35:08.705034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:08.705008    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" (UID: "28c090e2-209d-4a5d-8ae6-2a64ecbd09ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:35:08.805954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:08.805861    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:35:09.152369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:09.152335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:35:09.464778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:09.464686    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"
Jul 18 01:35:09.464778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:09.464727    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d" event={"ID":"28c090e2-209d-4a5d-8ae6-2a64ecbd09ed","Type":"ContainerDied","Data":"2819242b914e5f2ab2f8766ea10a21dbeb1219e3ea549b41d5110bb23f7026d8"}
Jul 18 01:35:09.464995 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:09.464792    2567 scope.go:117] "RemoveContainer" containerID="0e020781fb69c70f93cdd1df324077eb372b7ac396d15d19e0ef67a831fd136b"
Jul 18 01:35:09.487046 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:09.487009    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"]
Jul 18 01:35:09.490764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:09.490733    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-p652d"]
Jul 18 01:35:10.916028 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:10.916003    2567 scope.go:117] "RemoveContainer" containerID="1e20ecd1ab5694370775192a0a69eef2d598754dc31f7ff5785d0d99e55afacb"
Jul 18 01:35:10.920623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:10.920594    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" path="/var/lib/kubelet/pods/28c090e2-209d-4a5d-8ae6-2a64ecbd09ed/volumes"
Jul 18 01:35:11.479521 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:11.479440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" event={"ID":"2a3d52bb-b651-4b68-a671-1433c4277552","Type":"ContainerStarted","Data":"e5d396b5929e2be1014491a0865c9f43798b9fb578ce3fbed98b3fbbf80d9637"}
Jul 18 01:35:11.509582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:11.509523    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" podStartSLOduration=20.509506986 podStartE2EDuration="20.509506986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:35:11.507471004 +0000 UTC m=+3291.034975967" watchObservedRunningTime="2026-07-18 01:35:11.509506986 +0000 UTC m=+3291.037011975"
Jul 18 01:35:12.487740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:12.487702    2567 generic.go:356] "Generic (PLEG): container finished" podID="2a3d52bb-b651-4b68-a671-1433c4277552" containerID="e5d396b5929e2be1014491a0865c9f43798b9fb578ce3fbed98b3fbbf80d9637" exitCode=6
Jul 18 01:35:12.488227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:12.487777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" event={"ID":"2a3d52bb-b651-4b68-a671-1433c4277552","Type":"ContainerDied","Data":"e5d396b5929e2be1014491a0865c9f43798b9fb578ce3fbed98b3fbbf80d9637"}
Jul 18 01:35:12.488227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:12.487837    2567 scope.go:117] "RemoveContainer" containerID="1e20ecd1ab5694370775192a0a69eef2d598754dc31f7ff5785d0d99e55afacb"
Jul 18 01:35:12.488227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:12.488179    2567 scope.go:117] "RemoveContainer" containerID="e5d396b5929e2be1014491a0865c9f43798b9fb578ce3fbed98b3fbbf80d9637"
Jul 18 01:35:12.488492 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:12.488469    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq_odh-ai-gateway-infra(2a3d52bb-b651-4b68-a671-1433c4277552)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" podUID="2a3d52bb-b651-4b68-a671-1433c4277552"
Jul 18 01:35:12.516786 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:12.516762    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"]
Jul 18 01:35:13.310093 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:13.310054    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:35:13.639455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:13.639428    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:35:13.753451 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:13.753416    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3d52bb-b651-4b68-a671-1433c4277552-maas-api-service-ca\") pod \"2a3d52bb-b651-4b68-a671-1433c4277552\" (UID: \"2a3d52bb-b651-4b68-a671-1433c4277552\") "
Jul 18 01:35:13.753789 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:13.753756    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a3d52bb-b651-4b68-a671-1433c4277552-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a3d52bb-b651-4b68-a671-1433c4277552" (UID: "2a3d52bb-b651-4b68-a671-1433c4277552"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:35:13.854260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:13.854223    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a3d52bb-b651-4b68-a671-1433c4277552-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:35:14.502685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:14.502651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq" event={"ID":"2a3d52bb-b651-4b68-a671-1433c4277552","Type":"ContainerDied","Data":"801ba5e8c999eafeee4e440c4c3871c3c8e50bfc722eec38c3b6dc54af3babc4"}
Jul 18 01:35:14.502685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:14.502673    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"
Jul 18 01:35:14.502962 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:14.502723    2567 scope.go:117] "RemoveContainer" containerID="e5d396b5929e2be1014491a0865c9f43798b9fb578ce3fbed98b3fbbf80d9637"
Jul 18 01:35:14.524221 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:14.524191    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"]
Jul 18 01:35:14.526392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:14.526368    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-ccqgq"]
Jul 18 01:35:14.925867 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:14.925827    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" path="/var/lib/kubelet/pods/2a3d52bb-b651-4b68-a671-1433c4277552/volumes"
Jul 18 01:35:21.143354 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:21.143323    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:35:21.148784 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:21.148760    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:35:28.809508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:28.809473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:35:34.044712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.044676    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"]
Jul 18 01:35:34.045132 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045118    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045135    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045143    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:34.045178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045152    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:34.045178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045165    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:34.045178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045170    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:34.045178 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045177    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045182    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045193    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045198    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045205    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045211    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045218    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045225    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045246    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045251    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045335    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045343    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045349    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045356    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045363    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045368    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:34.045394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.045376    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="534ae410-6b97-41d6-8b9b-418ae8b61a3b" containerName="revoke-keys"
Jul 18 01:35:34.048552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.048536    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:34.051181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.051139    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:35:34.051431 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.051415    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:35:34.058456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.058427    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"]
Jul 18 01:35:34.154863 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.154824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a006d3cb-368f-4581-bf00-ce35a7415df7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-64fsc\" (UID: \"a006d3cb-368f-4581-bf00-ce35a7415df7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:34.255626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.255594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a006d3cb-368f-4581-bf00-ce35a7415df7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-64fsc\" (UID: \"a006d3cb-368f-4581-bf00-ce35a7415df7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:34.256263 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.256238    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a006d3cb-368f-4581-bf00-ce35a7415df7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-64fsc\" (UID: \"a006d3cb-368f-4581-bf00-ce35a7415df7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:34.362074 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.362034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:34.510704 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.510673    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"]
Jul 18 01:35:34.512747 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:35:34.512713    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda006d3cb_368f_4581_bf00_ce35a7415df7.slice/crio-634a196b73fa9cf136535d1367c881eed500ebb1fce16ff3f8ac1b272a9c5ccf WatchSource:0}: Error finding container 634a196b73fa9cf136535d1367c881eed500ebb1fce16ff3f8ac1b272a9c5ccf: Status 404 returned error can't find the container with id 634a196b73fa9cf136535d1367c881eed500ebb1fce16ff3f8ac1b272a9c5ccf
Jul 18 01:35:34.614511 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:34.614422    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" event={"ID":"a006d3cb-368f-4581-bf00-ce35a7415df7","Type":"ContainerStarted","Data":"634a196b73fa9cf136535d1367c881eed500ebb1fce16ff3f8ac1b272a9c5ccf"}
Jul 18 01:35:35.621522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:35.621489    2567 generic.go:356] "Generic (PLEG): container finished" podID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerID="e735148050ff7fc60f409e5d274f14f886ea5711344882e73227961415cfb447" exitCode=6
Jul 18 01:35:35.621943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:35.621586    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" event={"ID":"a006d3cb-368f-4581-bf00-ce35a7415df7","Type":"ContainerDied","Data":"e735148050ff7fc60f409e5d274f14f886ea5711344882e73227961415cfb447"}
Jul 18 01:35:35.621943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:35.621759    2567 scope.go:117] "RemoveContainer" containerID="e735148050ff7fc60f409e5d274f14f886ea5711344882e73227961415cfb447"
Jul 18 01:35:36.629065 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:36.628967    2567 generic.go:356] "Generic (PLEG): container finished" podID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerID="da610585d2f118c5af70a5305a55213ad9636df02fc10d777d4052cc46956996" exitCode=6
Jul 18 01:35:36.629540 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:36.629063    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" event={"ID":"a006d3cb-368f-4581-bf00-ce35a7415df7","Type":"ContainerDied","Data":"da610585d2f118c5af70a5305a55213ad9636df02fc10d777d4052cc46956996"}
Jul 18 01:35:36.629540 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:36.629125    2567 scope.go:117] "RemoveContainer" containerID="e735148050ff7fc60f409e5d274f14f886ea5711344882e73227961415cfb447"
Jul 18 01:35:36.629540 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:36.629369    2567 scope.go:117] "RemoveContainer" containerID="da610585d2f118c5af70a5305a55213ad9636df02fc10d777d4052cc46956996"
Jul 18 01:35:36.629717 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:36.629630    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-64fsc_odh-ai-gateway-infra(a006d3cb-368f-4581-bf00-ce35a7415df7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7"
Jul 18 01:35:37.636071 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:37.636042    2567 scope.go:117] "RemoveContainer" containerID="da610585d2f118c5af70a5305a55213ad9636df02fc10d777d4052cc46956996"
Jul 18 01:35:37.636535 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:37.636342    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-64fsc_odh-ai-gateway-infra(a006d3cb-368f-4581-bf00-ce35a7415df7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7"
Jul 18 01:35:46.309176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:46.309098    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"]
Jul 18 01:35:46.327077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:46.327042    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:35:47.202678 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:47.202642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:35:49.452169 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.452127    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"]
Jul 18 01:35:49.452669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.452539    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:49.452669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.452550    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:49.452669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.452622    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c090e2-209d-4a5d-8ae6-2a64ecbd09ed" containerName="revoke-keys"
Jul 18 01:35:49.452669 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.452632    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3d52bb-b651-4b68-a671-1433c4277552" containerName="revoke-keys"
Jul 18 01:35:49.457095 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.457072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:35:49.463518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.463496    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"]
Jul 18 01:35:49.482476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.482444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17ee0bab-010f-4867-b847-08dcf75c9c64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8\" (UID: \"17ee0bab-010f-4867-b847-08dcf75c9c64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:35:49.583449 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.583409    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17ee0bab-010f-4867-b847-08dcf75c9c64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8\" (UID: \"17ee0bab-010f-4867-b847-08dcf75c9c64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:35:49.584020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.583998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17ee0bab-010f-4867-b847-08dcf75c9c64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8\" (UID: \"17ee0bab-010f-4867-b847-08dcf75c9c64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:35:49.770307 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.770204    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:35:49.910670 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:49.910640    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"]
Jul 18 01:35:49.912022 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:35:49.911989    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ee0bab_010f_4867_b847_08dcf75c9c64.slice/crio-7daed517037f2ac5bce30dba6873aa695ed60299a1b4adb80a44d97225ae1ed8 WatchSource:0}: Error finding container 7daed517037f2ac5bce30dba6873aa695ed60299a1b4adb80a44d97225ae1ed8: Status 404 returned error can't find the container with id 7daed517037f2ac5bce30dba6873aa695ed60299a1b4adb80a44d97225ae1ed8
Jul 18 01:35:50.201938 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:50.201844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:35:50.706455 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:50.706419    2567 generic.go:356] "Generic (PLEG): container finished" podID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerID="1bae4b5c8d05dc4a65ad82fef341331e04d8ab40ead7a399d5bb9aac2246a6ed" exitCode=6
Jul 18 01:35:50.706859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:50.706506    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" event={"ID":"17ee0bab-010f-4867-b847-08dcf75c9c64","Type":"ContainerDied","Data":"1bae4b5c8d05dc4a65ad82fef341331e04d8ab40ead7a399d5bb9aac2246a6ed"}
Jul 18 01:35:50.706859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:50.706567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" event={"ID":"17ee0bab-010f-4867-b847-08dcf75c9c64","Type":"ContainerStarted","Data":"7daed517037f2ac5bce30dba6873aa695ed60299a1b4adb80a44d97225ae1ed8"}
Jul 18 01:35:50.706859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:50.706693    2567 scope.go:117] "RemoveContainer" containerID="1bae4b5c8d05dc4a65ad82fef341331e04d8ab40ead7a399d5bb9aac2246a6ed"
Jul 18 01:35:51.713806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:51.713700    2567 generic.go:356] "Generic (PLEG): container finished" podID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerID="68eca885abe9f3e0a84c3bb6c57aee92c74addfae9ee9efedcdb7b4e297a977e" exitCode=6
Jul 18 01:35:51.713806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:51.713754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" event={"ID":"17ee0bab-010f-4867-b847-08dcf75c9c64","Type":"ContainerDied","Data":"68eca885abe9f3e0a84c3bb6c57aee92c74addfae9ee9efedcdb7b4e297a977e"}
Jul 18 01:35:51.714357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:51.713817    2567 scope.go:117] "RemoveContainer" containerID="1bae4b5c8d05dc4a65ad82fef341331e04d8ab40ead7a399d5bb9aac2246a6ed"
Jul 18 01:35:51.714357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:51.714070    2567 scope.go:117] "RemoveContainer" containerID="68eca885abe9f3e0a84c3bb6c57aee92c74addfae9ee9efedcdb7b4e297a977e"
Jul 18 01:35:51.714449 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:51.714348    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8_odh-ai-gateway-infra(17ee0bab-010f-4867-b847-08dcf75c9c64)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64"
Jul 18 01:35:51.913240 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:51.913210    2567 scope.go:117] "RemoveContainer" containerID="da610585d2f118c5af70a5305a55213ad9636df02fc10d777d4052cc46956996"
Jul 18 01:35:52.126523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.126493    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"]
Jul 18 01:35:52.130076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.130055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.132489 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.132469    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6b3233\"/\"rate-test-model-6b3233-kserve-self-signed-certs\""
Jul 18 01:35:52.140542 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.140516    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"]
Jul 18 01:35:52.206407 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.206377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-home\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.206587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.206421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-dshm\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.206587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.206443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-model-cache\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.206587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.206505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e144c0-acb4-43dc-9d6a-db02c21944ff-tls-certs\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.206587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.206555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-tmp-dir\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.206778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.206587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv4k7\" (UniqueName: \"kubernetes.io/projected/29e144c0-acb4-43dc-9d6a-db02c21944ff-kube-api-access-sv4k7\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.215298 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.215258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:35:52.307094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-dshm\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-model-cache\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e144c0-acb4-43dc-9d6a-db02c21944ff-tls-certs\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-tmp-dir\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307195    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv4k7\" (UniqueName: \"kubernetes.io/projected/29e144c0-acb4-43dc-9d6a-db02c21944ff-kube-api-access-sv4k7\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307379 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-home\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-model-cache\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307679 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307633    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-tmp-dir\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.307679 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.307672    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-home\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.309589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.309572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-dshm\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.309666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.309652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e144c0-acb4-43dc-9d6a-db02c21944ff-tls-certs\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.316885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.316863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv4k7\" (UniqueName: \"kubernetes.io/projected/29e144c0-acb4-43dc-9d6a-db02c21944ff-kube-api-access-sv4k7\") pod \"rate-test-model-6b3233-kserve-85747f7d75-6gsxd\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") " pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.440987 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.440942    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:35:52.583051 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.583023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"]
Jul 18 01:35:52.596148 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:35:52.596120    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29e144c0_acb4_43dc_9d6a_db02c21944ff.slice/crio-65d39559135b3f331f6d6bdad781738721d085905ff0af46bc8ef1bb45361d7f WatchSource:0}: Error finding container 65d39559135b3f331f6d6bdad781738721d085905ff0af46bc8ef1bb45361d7f: Status 404 returned error can't find the container with id 65d39559135b3f331f6d6bdad781738721d085905ff0af46bc8ef1bb45361d7f
Jul 18 01:35:52.721312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.721260    2567 generic.go:356] "Generic (PLEG): container finished" podID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerID="7ba23683fe8da6c4806351c95a3636f449f670feb29cc5f54bfc5f80f4be274c" exitCode=6
Jul 18 01:35:52.721312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.721303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" event={"ID":"a006d3cb-368f-4581-bf00-ce35a7415df7","Type":"ContainerDied","Data":"7ba23683fe8da6c4806351c95a3636f449f670feb29cc5f54bfc5f80f4be274c"}
Jul 18 01:35:52.721865 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.721381    2567 scope.go:117] "RemoveContainer" containerID="da610585d2f118c5af70a5305a55213ad9636df02fc10d777d4052cc46956996"
Jul 18 01:35:52.721865 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.721683    2567 scope.go:117] "RemoveContainer" containerID="7ba23683fe8da6c4806351c95a3636f449f670feb29cc5f54bfc5f80f4be274c"
Jul 18 01:35:52.721987 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:52.721967    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-64fsc_odh-ai-gateway-infra(a006d3cb-368f-4581-bf00-ce35a7415df7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7"
Jul 18 01:35:52.723514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.723495    2567 scope.go:117] "RemoveContainer" containerID="68eca885abe9f3e0a84c3bb6c57aee92c74addfae9ee9efedcdb7b4e297a977e"
Jul 18 01:35:52.723733 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:52.723714    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8_odh-ai-gateway-infra(17ee0bab-010f-4867-b847-08dcf75c9c64)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64"
Jul 18 01:35:52.724944 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.724911    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd" event={"ID":"29e144c0-acb4-43dc-9d6a-db02c21944ff","Type":"ContainerStarted","Data":"6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18"}
Jul 18 01:35:52.725038 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.724950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd" event={"ID":"29e144c0-acb4-43dc-9d6a-db02c21944ff","Type":"ContainerStarted","Data":"65d39559135b3f331f6d6bdad781738721d085905ff0af46bc8ef1bb45361d7f"}
Jul 18 01:35:52.768889 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:52.768834    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd" podStartSLOduration=0.768817019 podStartE2EDuration="768.817019ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:35:52.767039674 +0000 UTC m=+3332.294544634" watchObservedRunningTime="2026-07-18 01:35:52.768817019 +0000 UTC m=+3332.296321978"
Jul 18 01:35:53.747301 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:53.747250    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"]
Jul 18 01:35:53.897470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:53.897447    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:53.923334 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:53.923307    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a006d3cb-368f-4581-bf00-ce35a7415df7-maas-api-service-ca\") pod \"a006d3cb-368f-4581-bf00-ce35a7415df7\" (UID: \"a006d3cb-368f-4581-bf00-ce35a7415df7\") "
Jul 18 01:35:53.923645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:53.923621    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a006d3cb-368f-4581-bf00-ce35a7415df7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a006d3cb-368f-4581-bf00-ce35a7415df7" (UID: "a006d3cb-368f-4581-bf00-ce35a7415df7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:35:54.024991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.024903    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a006d3cb-368f-4581-bf00-ce35a7415df7-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:35:54.744936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.744904    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"
Jul 18 01:35:54.745119 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.744938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc" event={"ID":"a006d3cb-368f-4581-bf00-ce35a7415df7","Type":"ContainerDied","Data":"634a196b73fa9cf136535d1367c881eed500ebb1fce16ff3f8ac1b272a9c5ccf"}
Jul 18 01:35:54.745119 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.745023    2567 scope.go:117] "RemoveContainer" containerID="7ba23683fe8da6c4806351c95a3636f449f670feb29cc5f54bfc5f80f4be274c"
Jul 18 01:35:54.767929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.767900    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"]
Jul 18 01:35:54.771189 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.771159    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-64fsc"]
Jul 18 01:35:54.919717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:54.919677    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" path="/var/lib/kubelet/pods/a006d3cb-368f-4581-bf00-ce35a7415df7/volumes"
Jul 18 01:35:56.687459 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687409    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"]
Jul 18 01:35:56.687849 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687816    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.687849 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687827    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.687849 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687842    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.687849 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687848    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.688035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687861    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.688035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687867    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.688035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687933    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.688035 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.687941    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:35:56.692606 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.692586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:35:56.698518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.698494    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"]
Jul 18 01:35:56.750980 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.750948    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1acf06a7-3eb1-4356-84d4-47da2f912cdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-272qc\" (UID: \"1acf06a7-3eb1-4356-84d4-47da2f912cdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:35:56.851986 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.851944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1acf06a7-3eb1-4356-84d4-47da2f912cdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-272qc\" (UID: \"1acf06a7-3eb1-4356-84d4-47da2f912cdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:35:56.852661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:56.852643    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1acf06a7-3eb1-4356-84d4-47da2f912cdc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-272qc\" (UID: \"1acf06a7-3eb1-4356-84d4-47da2f912cdc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:35:57.007004 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:57.006918    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:35:57.146123 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:57.146091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"]
Jul 18 01:35:57.147325 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:35:57.147298    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1acf06a7_3eb1_4356_84d4_47da2f912cdc.slice/crio-11cbf6f6c18b963f0d02049382b5fdc1ca70a3676b941aea7d5391d76dd091e6 WatchSource:0}: Error finding container 11cbf6f6c18b963f0d02049382b5fdc1ca70a3676b941aea7d5391d76dd091e6: Status 404 returned error can't find the container with id 11cbf6f6c18b963f0d02049382b5fdc1ca70a3676b941aea7d5391d76dd091e6
Jul 18 01:35:57.766802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:57.766768    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" event={"ID":"1acf06a7-3eb1-4356-84d4-47da2f912cdc","Type":"ContainerStarted","Data":"2c4b41f07cca5bdb2609d4943cda7eb50b5dbb494e8f2d92e39df0aa22d5e6e0"}
Jul 18 01:35:57.767212 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:57.766837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" event={"ID":"1acf06a7-3eb1-4356-84d4-47da2f912cdc","Type":"ContainerStarted","Data":"11cbf6f6c18b963f0d02049382b5fdc1ca70a3676b941aea7d5391d76dd091e6"}
Jul 18 01:35:57.782540 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:57.782494    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" podStartSLOduration=1.782479247 podStartE2EDuration="1.782479247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:35:57.780396288 +0000 UTC m=+3337.307901247" watchObservedRunningTime="2026-07-18 01:35:57.782479247 +0000 UTC m=+3337.309984246"
Jul 18 01:35:58.774884 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:58.774846    2567 generic.go:356] "Generic (PLEG): container finished" podID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerID="2c4b41f07cca5bdb2609d4943cda7eb50b5dbb494e8f2d92e39df0aa22d5e6e0" exitCode=6
Jul 18 01:35:58.775414 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:58.774932    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" event={"ID":"1acf06a7-3eb1-4356-84d4-47da2f912cdc","Type":"ContainerDied","Data":"2c4b41f07cca5bdb2609d4943cda7eb50b5dbb494e8f2d92e39df0aa22d5e6e0"}
Jul 18 01:35:58.775414 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:58.775126    2567 scope.go:117] "RemoveContainer" containerID="2c4b41f07cca5bdb2609d4943cda7eb50b5dbb494e8f2d92e39df0aa22d5e6e0"
Jul 18 01:35:59.782229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:59.782150    2567 generic.go:356] "Generic (PLEG): container finished" podID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerID="50151292efa4411f5780db296c7f1b0a7c1db06cdda7162c5f868aad479d2856" exitCode=6
Jul 18 01:35:59.782649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:59.782224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" event={"ID":"1acf06a7-3eb1-4356-84d4-47da2f912cdc","Type":"ContainerDied","Data":"50151292efa4411f5780db296c7f1b0a7c1db06cdda7162c5f868aad479d2856"}
Jul 18 01:35:59.782649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:59.782296    2567 scope.go:117] "RemoveContainer" containerID="2c4b41f07cca5bdb2609d4943cda7eb50b5dbb494e8f2d92e39df0aa22d5e6e0"
Jul 18 01:35:59.782649 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:35:59.782500    2567 scope.go:117] "RemoveContainer" containerID="50151292efa4411f5780db296c7f1b0a7c1db06cdda7162c5f868aad479d2856"
Jul 18 01:35:59.782772 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:35:59.782736    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-272qc_odh-ai-gateway-infra(1acf06a7-3eb1-4356-84d4-47da2f912cdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc"
Jul 18 01:36:00.789731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:00.789705    2567 scope.go:117] "RemoveContainer" containerID="50151292efa4411f5780db296c7f1b0a7c1db06cdda7162c5f868aad479d2856"
Jul 18 01:36:00.790218 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:00.789922    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-272qc_odh-ai-gateway-infra(1acf06a7-3eb1-4356-84d4-47da2f912cdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc"
Jul 18 01:36:01.330588 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.330527    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs" podUID="90b1afdb-a84d-4f33-b207-ed4731d9c12c" containerName="main" containerID="cri-o://f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380" gracePeriod=45
Jul 18 01:36:01.600328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.600302    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:36:01.700302 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700232    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tls-certs\") pod \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") "
Jul 18 01:36:01.700302 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700302    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-home\") pod \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") "
Jul 18 01:36:01.700302 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700321    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tmp-dir\") pod \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") "
Jul 18 01:36:01.700646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700339    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p22cz\" (UniqueName: \"kubernetes.io/projected/90b1afdb-a84d-4f33-b207-ed4731d9c12c-kube-api-access-p22cz\") pod \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") "
Jul 18 01:36:01.700646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700375    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-model-cache\") pod \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") "
Jul 18 01:36:01.700646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700424    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-dshm\") pod \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\" (UID: \"90b1afdb-a84d-4f33-b207-ed4731d9c12c\") "
Jul 18 01:36:01.700646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700603    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-home" (OuterVolumeSpecName: "home") pod "90b1afdb-a84d-4f33-b207-ed4731d9c12c" (UID: "90b1afdb-a84d-4f33-b207-ed4731d9c12c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:36:01.700868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700618    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90b1afdb-a84d-4f33-b207-ed4731d9c12c" (UID: "90b1afdb-a84d-4f33-b207-ed4731d9c12c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:36:01.700868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700706    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-model-cache" (OuterVolumeSpecName: "model-cache") pod "90b1afdb-a84d-4f33-b207-ed4731d9c12c" (UID: "90b1afdb-a84d-4f33-b207-ed4731d9c12c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:36:01.700868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700812    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:01.700868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700833    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:01.700868 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.700846    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:01.702731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.702707    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-dshm" (OuterVolumeSpecName: "dshm") pod "90b1afdb-a84d-4f33-b207-ed4731d9c12c" (UID: "90b1afdb-a84d-4f33-b207-ed4731d9c12c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:36:01.702828 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.702765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b1afdb-a84d-4f33-b207-ed4731d9c12c-kube-api-access-p22cz" (OuterVolumeSpecName: "kube-api-access-p22cz") pod "90b1afdb-a84d-4f33-b207-ed4731d9c12c" (UID: "90b1afdb-a84d-4f33-b207-ed4731d9c12c"). InnerVolumeSpecName "kube-api-access-p22cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:36:01.702882 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.702833    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90b1afdb-a84d-4f33-b207-ed4731d9c12c" (UID: "90b1afdb-a84d-4f33-b207-ed4731d9c12c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:36:01.798719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.798686    2567 generic.go:356] "Generic (PLEG): container finished" podID="90b1afdb-a84d-4f33-b207-ed4731d9c12c" containerID="f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380" exitCode=0
Jul 18 01:36:01.799100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.798739    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"
Jul 18 01:36:01.799100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.798769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs" event={"ID":"90b1afdb-a84d-4f33-b207-ed4731d9c12c","Type":"ContainerDied","Data":"f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380"}
Jul 18 01:36:01.799100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.798822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs" event={"ID":"90b1afdb-a84d-4f33-b207-ed4731d9c12c","Type":"ContainerDied","Data":"8ef38e591ae9b6d7dcfaa04a5c6f618563510395926ab130a469e8928a4102b3"}
Jul 18 01:36:01.799100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.798842    2567 scope.go:117] "RemoveContainer" containerID="f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380"
Jul 18 01:36:01.801763 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.801742    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90b1afdb-a84d-4f33-b207-ed4731d9c12c-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:01.801871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.801766    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90b1afdb-a84d-4f33-b207-ed4731d9c12c-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:01.801871 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.801786    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p22cz\" (UniqueName: \"kubernetes.io/projected/90b1afdb-a84d-4f33-b207-ed4731d9c12c-kube-api-access-p22cz\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:01.811675 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.811653    2567 scope.go:117] "RemoveContainer" containerID="f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380"
Jul 18 01:36:01.811972 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:01.811952    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380\": container with ID starting with f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380 not found: ID does not exist" containerID="f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380"
Jul 18 01:36:01.812022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.811986    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380"} err="failed to get container status \"f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380\": rpc error: code = NotFound desc = could not find container \"f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380\": container with ID starting with f3bdeda7b60031e4bf4488eac991747ea0698d28880f9ffdd6ef6f8240659380 not found: ID does not exist"
Jul 18 01:36:01.856351 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.856249    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"]
Jul 18 01:36:01.859810 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:01.859780    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/sub-test-model-6b3233-kserve-86564958bf-7zqcs"]
Jul 18 01:36:02.441082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:02.441046    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:36:02.441303 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:02.441104    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:36:02.456079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:02.456049    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:36:02.816806 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:02.816780    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:36:02.919286 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:02.919231    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b1afdb-a84d-4f33-b207-ed4731d9c12c" path="/var/lib/kubelet/pods/90b1afdb-a84d-4f33-b207-ed4731d9c12c/volumes"
Jul 18 01:36:06.914113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:06.914079    2567 scope.go:117] "RemoveContainer" containerID="68eca885abe9f3e0a84c3bb6c57aee92c74addfae9ee9efedcdb7b4e297a977e"
Jul 18 01:36:07.699072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:07.699033    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:36:07.836397 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:07.836361    2567 generic.go:356] "Generic (PLEG): container finished" podID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerID="2555136a9292188f050a4265ae587d89fd859fe6e4bcc8742d1d9721d0ec248f" exitCode=6
Jul 18 01:36:07.836607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:07.836436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" event={"ID":"17ee0bab-010f-4867-b847-08dcf75c9c64","Type":"ContainerDied","Data":"2555136a9292188f050a4265ae587d89fd859fe6e4bcc8742d1d9721d0ec248f"}
Jul 18 01:36:07.836607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:07.836509    2567 scope.go:117] "RemoveContainer" containerID="68eca885abe9f3e0a84c3bb6c57aee92c74addfae9ee9efedcdb7b4e297a977e"
Jul 18 01:36:07.836964 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:07.836939    2567 scope.go:117] "RemoveContainer" containerID="2555136a9292188f050a4265ae587d89fd859fe6e4bcc8742d1d9721d0ec248f"
Jul 18 01:36:07.837235 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:07.837212    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8_odh-ai-gateway-infra(17ee0bab-010f-4867-b847-08dcf75c9c64)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64"
Jul 18 01:36:08.861889 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:08.861850    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"]
Jul 18 01:36:09.005486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.005455    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:36:09.071475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.071439    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17ee0bab-010f-4867-b847-08dcf75c9c64-maas-api-service-ca\") pod \"17ee0bab-010f-4867-b847-08dcf75c9c64\" (UID: \"17ee0bab-010f-4867-b847-08dcf75c9c64\") "
Jul 18 01:36:09.071859 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.071831    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17ee0bab-010f-4867-b847-08dcf75c9c64-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17ee0bab-010f-4867-b847-08dcf75c9c64" (UID: "17ee0bab-010f-4867-b847-08dcf75c9c64"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:36:09.172509 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.172422    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17ee0bab-010f-4867-b847-08dcf75c9c64-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:09.377811 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.377773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:09.477988 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.477901    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:09.850918 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.850890    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"
Jul 18 01:36:09.851135 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.850890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8" event={"ID":"17ee0bab-010f-4867-b847-08dcf75c9c64","Type":"ContainerDied","Data":"7daed517037f2ac5bce30dba6873aa695ed60299a1b4adb80a44d97225ae1ed8"}
Jul 18 01:36:09.851135 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.851021    2567 scope.go:117] "RemoveContainer" containerID="2555136a9292188f050a4265ae587d89fd859fe6e4bcc8742d1d9721d0ec248f"
Jul 18 01:36:09.873441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.873410    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"]
Jul 18 01:36:09.878327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:09.878300    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-ktwr8"]
Jul 18 01:36:10.919778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:10.919744    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" path="/var/lib/kubelet/pods/17ee0bab-010f-4867-b847-08dcf75c9c64/volumes"
Jul 18 01:36:12.913337 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:12.913304    2567 scope.go:117] "RemoveContainer" containerID="50151292efa4411f5780db296c7f1b0a7c1db06cdda7162c5f868aad479d2856"
Jul 18 01:36:13.877764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:13.877735    2567 generic.go:356] "Generic (PLEG): container finished" podID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerID="b2ce2b921b7adf9539cb2b8e4b44913323150e7ed6716dfd894301d0fbc4d20b" exitCode=6
Jul 18 01:36:13.877982 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:13.877813    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" event={"ID":"1acf06a7-3eb1-4356-84d4-47da2f912cdc","Type":"ContainerDied","Data":"b2ce2b921b7adf9539cb2b8e4b44913323150e7ed6716dfd894301d0fbc4d20b"}
Jul 18 01:36:13.877982 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:13.877889    2567 scope.go:117] "RemoveContainer" containerID="50151292efa4411f5780db296c7f1b0a7c1db06cdda7162c5f868aad479d2856"
Jul 18 01:36:13.878166 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:13.878149    2567 scope.go:117] "RemoveContainer" containerID="b2ce2b921b7adf9539cb2b8e4b44913323150e7ed6716dfd894301d0fbc4d20b"
Jul 18 01:36:13.878473 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:13.878448    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-272qc_odh-ai-gateway-infra(1acf06a7-3eb1-4356-84d4-47da2f912cdc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc"
Jul 18 01:36:14.905656 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:14.905609    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"]
Jul 18 01:36:15.053226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.053200    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:36:15.130197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.130161    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1acf06a7-3eb1-4356-84d4-47da2f912cdc-maas-api-service-ca\") pod \"1acf06a7-3eb1-4356-84d4-47da2f912cdc\" (UID: \"1acf06a7-3eb1-4356-84d4-47da2f912cdc\") "
Jul 18 01:36:15.130559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.130537    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1acf06a7-3eb1-4356-84d4-47da2f912cdc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1acf06a7-3eb1-4356-84d4-47da2f912cdc" (UID: "1acf06a7-3eb1-4356-84d4-47da2f912cdc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:36:15.231374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.231251    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1acf06a7-3eb1-4356-84d4-47da2f912cdc-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:15.891658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.891626    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"
Jul 18 01:36:15.891658 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.891634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc" event={"ID":"1acf06a7-3eb1-4356-84d4-47da2f912cdc","Type":"ContainerDied","Data":"11cbf6f6c18b963f0d02049382b5fdc1ca70a3676b941aea7d5391d76dd091e6"}
Jul 18 01:36:15.891937 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.891689    2567 scope.go:117] "RemoveContainer" containerID="b2ce2b921b7adf9539cb2b8e4b44913323150e7ed6716dfd894301d0fbc4d20b"
Jul 18 01:36:15.915373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.915339    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"]
Jul 18 01:36:15.923764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:15.923734    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-272qc"]
Jul 18 01:36:16.919541 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:16.919508    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" path="/var/lib/kubelet/pods/1acf06a7-3eb1-4356-84d4-47da2f912cdc/volumes"
Jul 18 01:36:21.756127 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:21.756086    2567 scope.go:117] "RemoveContainer" containerID="6f17192880126b69df39f317d54e1f49aebb274b46cbaacace138888f870323c"
Jul 18 01:36:34.073164 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.073128    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:34.116133 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116094    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"]
Jul 18 01:36:34.116518 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116504    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116520    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116529    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116534    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116542    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116549    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116557    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b1afdb-a84d-4f33-b207-ed4731d9c12c" containerName="main"
Jul 18 01:36:34.116568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116563    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b1afdb-a84d-4f33-b207-ed4731d9c12c" containerName="main"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116572    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116577    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116648    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116656    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116663    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116671    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116676    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a006d3cb-368f-4581-bf00-ce35a7415df7" containerName="revoke-keys"
Jul 18 01:36:34.116817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.116683    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b1afdb-a84d-4f33-b207-ed4731d9c12c" containerName="main"
Jul 18 01:36:34.119716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.119697    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:34.122136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.122107    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:36:34.122136 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.122125    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:36:34.128037 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.128009    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"]
Jul 18 01:36:34.145800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.145761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:34.209013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.208965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abfdf46-caf5-4e8b-a011-a3ac96893119-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-47mv8\" (UID: \"9abfdf46-caf5-4e8b-a011-a3ac96893119\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:34.310062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.310019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abfdf46-caf5-4e8b-a011-a3ac96893119-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-47mv8\" (UID: \"9abfdf46-caf5-4e8b-a011-a3ac96893119\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:34.310705 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.310682    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abfdf46-caf5-4e8b-a011-a3ac96893119-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-47mv8\" (UID: \"9abfdf46-caf5-4e8b-a011-a3ac96893119\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:34.433408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.433301    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:34.573853 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.573822    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"]
Jul 18 01:36:34.576787 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:36:34.576756    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abfdf46_caf5_4e8b_a011_a3ac96893119.slice/crio-0c07c39e5b84d6f82e13d2abd721aa51e587c6d955882ffeeaf2608ea32658c9 WatchSource:0}: Error finding container 0c07c39e5b84d6f82e13d2abd721aa51e587c6d955882ffeeaf2608ea32658c9: Status 404 returned error can't find the container with id 0c07c39e5b84d6f82e13d2abd721aa51e587c6d955882ffeeaf2608ea32658c9
Jul 18 01:36:34.999239 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.999199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" event={"ID":"9abfdf46-caf5-4e8b-a011-a3ac96893119","Type":"ContainerStarted","Data":"8f0fb8df4f9bb3445294338bb45e05271a48643c48e85720a8e63aafe6144b7f"}
Jul 18 01:36:34.999439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:34.999253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" event={"ID":"9abfdf46-caf5-4e8b-a011-a3ac96893119","Type":"ContainerStarted","Data":"0c07c39e5b84d6f82e13d2abd721aa51e587c6d955882ffeeaf2608ea32658c9"}
Jul 18 01:36:35.014798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:35.014747    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" podStartSLOduration=1.014729502 podStartE2EDuration="1.014729502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:36:35.013111738 +0000 UTC m=+3374.540616700" watchObservedRunningTime="2026-07-18 01:36:35.014729502 +0000 UTC m=+3374.542234462"
Jul 18 01:36:36.006445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:36.006415    2567 generic.go:356] "Generic (PLEG): container finished" podID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerID="8f0fb8df4f9bb3445294338bb45e05271a48643c48e85720a8e63aafe6144b7f" exitCode=6
Jul 18 01:36:36.006947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:36.006514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" event={"ID":"9abfdf46-caf5-4e8b-a011-a3ac96893119","Type":"ContainerDied","Data":"8f0fb8df4f9bb3445294338bb45e05271a48643c48e85720a8e63aafe6144b7f"}
Jul 18 01:36:36.006947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:36.006715    2567 scope.go:117] "RemoveContainer" containerID="8f0fb8df4f9bb3445294338bb45e05271a48643c48e85720a8e63aafe6144b7f"
Jul 18 01:36:37.014023 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:37.013929    2567 generic.go:356] "Generic (PLEG): container finished" podID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerID="74703f4c92ad243faf84f17ba79217303b291ae6ab11cae64ba08debb27f5b44" exitCode=6
Jul 18 01:36:37.014458 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:37.014015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" event={"ID":"9abfdf46-caf5-4e8b-a011-a3ac96893119","Type":"ContainerDied","Data":"74703f4c92ad243faf84f17ba79217303b291ae6ab11cae64ba08debb27f5b44"}
Jul 18 01:36:37.014458 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:37.014075    2567 scope.go:117] "RemoveContainer" containerID="8f0fb8df4f9bb3445294338bb45e05271a48643c48e85720a8e63aafe6144b7f"
Jul 18 01:36:37.014458 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:37.014249    2567 scope.go:117] "RemoveContainer" containerID="74703f4c92ad243faf84f17ba79217303b291ae6ab11cae64ba08debb27f5b44"
Jul 18 01:36:37.014576 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:37.014512    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-47mv8_odh-ai-gateway-infra(9abfdf46-caf5-4e8b-a011-a3ac96893119)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119"
Jul 18 01:36:38.021450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:38.021423    2567 scope.go:117] "RemoveContainer" containerID="74703f4c92ad243faf84f17ba79217303b291ae6ab11cae64ba08debb27f5b44"
Jul 18 01:36:38.021844 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:38.021613    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-47mv8_odh-ai-gateway-infra(9abfdf46-caf5-4e8b-a011-a3ac96893119)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119"
Jul 18 01:36:40.934570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:40.934535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:43.136774 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:43.136744    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:43.236885 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:43.236849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:36:45.210516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:45.210472    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:36:46.732473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.732436    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"]
Jul 18 01:36:46.733118 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.733095    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:46.733226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.733122    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:46.733226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.733144    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:46.733226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.733153    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:46.733374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.733283    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1acf06a7-3eb1-4356-84d4-47da2f912cdc" containerName="revoke-keys"
Jul 18 01:36:46.733374 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.733307    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ee0bab-010f-4867-b847-08dcf75c9c64" containerName="revoke-keys"
Jul 18 01:36:46.738327 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.738301    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:36:46.741318 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.741259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"]
Jul 18 01:36:46.827201 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.827154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f690d2-03a1-4729-a745-0ab5b40cf28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-665g5\" (UID: \"05f690d2-03a1-4729-a745-0ab5b40cf28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:36:46.928473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.928439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f690d2-03a1-4729-a745-0ab5b40cf28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-665g5\" (UID: \"05f690d2-03a1-4729-a745-0ab5b40cf28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:36:46.929056 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:46.929034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f690d2-03a1-4729-a745-0ab5b40cf28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-665g5\" (UID: \"05f690d2-03a1-4729-a745-0ab5b40cf28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:36:47.052308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:47.052200    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:36:47.195537 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:47.195508    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"]
Jul 18 01:36:47.197021 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:36:47.196989    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f690d2_03a1_4729_a745_0ab5b40cf28e.slice/crio-cd1c9d2a28f17ec252435a684fb1b29403394572b8eea60498db3609a8f9729b WatchSource:0}: Error finding container cd1c9d2a28f17ec252435a684fb1b29403394572b8eea60498db3609a8f9729b: Status 404 returned error can't find the container with id cd1c9d2a28f17ec252435a684fb1b29403394572b8eea60498db3609a8f9729b
Jul 18 01:36:48.081580 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:48.081542    2567 generic.go:356] "Generic (PLEG): container finished" podID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerID="476509ef91454e6d8ac2c4cf9cf3dcc59fa0efe7de29f6f7fe82b1c47426da09" exitCode=6
Jul 18 01:36:48.082098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:48.081630    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" event={"ID":"05f690d2-03a1-4729-a745-0ab5b40cf28e","Type":"ContainerDied","Data":"476509ef91454e6d8ac2c4cf9cf3dcc59fa0efe7de29f6f7fe82b1c47426da09"}
Jul 18 01:36:48.082098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:48.081683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" event={"ID":"05f690d2-03a1-4729-a745-0ab5b40cf28e","Type":"ContainerStarted","Data":"cd1c9d2a28f17ec252435a684fb1b29403394572b8eea60498db3609a8f9729b"}
Jul 18 01:36:48.082098 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:48.081831    2567 scope.go:117] "RemoveContainer" containerID="476509ef91454e6d8ac2c4cf9cf3dcc59fa0efe7de29f6f7fe82b1c47426da09"
Jul 18 01:36:49.089715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:49.089683    2567 generic.go:356] "Generic (PLEG): container finished" podID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerID="aef0d797150954a752ba26f67cacdaf54f065e499693e1f99ac70a78bd542f13" exitCode=6
Jul 18 01:36:49.090160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:49.089727    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" event={"ID":"05f690d2-03a1-4729-a745-0ab5b40cf28e","Type":"ContainerDied","Data":"aef0d797150954a752ba26f67cacdaf54f065e499693e1f99ac70a78bd542f13"}
Jul 18 01:36:49.090160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:49.089785    2567 scope.go:117] "RemoveContainer" containerID="476509ef91454e6d8ac2c4cf9cf3dcc59fa0efe7de29f6f7fe82b1c47426da09"
Jul 18 01:36:49.090160 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:49.089954    2567 scope.go:117] "RemoveContainer" containerID="aef0d797150954a752ba26f67cacdaf54f065e499693e1f99ac70a78bd542f13"
Jul 18 01:36:49.090316 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:49.090220    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-665g5_odh-ai-gateway-infra(05f690d2-03a1-4729-a745-0ab5b40cf28e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e"
Jul 18 01:36:50.096666 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:50.096619    2567 scope.go:117] "RemoveContainer" containerID="aef0d797150954a752ba26f67cacdaf54f065e499693e1f99ac70a78bd542f13"
Jul 18 01:36:50.097049 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:50.096906    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-665g5_odh-ai-gateway-infra(05f690d2-03a1-4729-a745-0ab5b40cf28e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e"
Jul 18 01:36:50.915403 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:50.915370    2567 scope.go:117] "RemoveContainer" containerID="74703f4c92ad243faf84f17ba79217303b291ae6ab11cae64ba08debb27f5b44"
Jul 18 01:36:51.668570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.668529    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"]
Jul 18 01:36:51.672043 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.672022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:36:51.681055 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.680825    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"]
Jul 18 01:36:51.776851 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.776753    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c37383-5e0d-4682-b4a4-a0210f7b9759-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq\" (UID: \"e9c37383-5e0d-4682-b4a4-a0210f7b9759\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:36:51.877585 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.877553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c37383-5e0d-4682-b4a4-a0210f7b9759-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq\" (UID: \"e9c37383-5e0d-4682-b4a4-a0210f7b9759\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:36:51.878342 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.878314    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c37383-5e0d-4682-b4a4-a0210f7b9759-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq\" (UID: \"e9c37383-5e0d-4682-b4a4-a0210f7b9759\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:36:51.985508 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:51.985478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:36:52.112143 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:52.112106    2567 generic.go:356] "Generic (PLEG): container finished" podID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerID="229e94368039d455cabacd9ffed6ed860ca12b0b1909e18293b74347a5b42fe4" exitCode=6
Jul 18 01:36:52.112385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:52.112179    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" event={"ID":"9abfdf46-caf5-4e8b-a011-a3ac96893119","Type":"ContainerDied","Data":"229e94368039d455cabacd9ffed6ed860ca12b0b1909e18293b74347a5b42fe4"}
Jul 18 01:36:52.112385 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:52.112252    2567 scope.go:117] "RemoveContainer" containerID="74703f4c92ad243faf84f17ba79217303b291ae6ab11cae64ba08debb27f5b44"
Jul 18 01:36:52.112826 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:52.112593    2567 scope.go:117] "RemoveContainer" containerID="229e94368039d455cabacd9ffed6ed860ca12b0b1909e18293b74347a5b42fe4"
Jul 18 01:36:52.112954 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:52.112836    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-47mv8_odh-ai-gateway-infra(9abfdf46-caf5-4e8b-a011-a3ac96893119)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119"
Jul 18 01:36:52.134936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:52.134905    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"]
Jul 18 01:36:52.135102 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:36:52.135076    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c37383_5e0d_4682_b4a4_a0210f7b9759.slice/crio-4c679bee2b80fef4788c051ce91729ad3c6fe1bad9d507bd438e601b130e243d WatchSource:0}: Error finding container 4c679bee2b80fef4788c051ce91729ad3c6fe1bad9d507bd438e601b130e243d: Status 404 returned error can't find the container with id 4c679bee2b80fef4788c051ce91729ad3c6fe1bad9d507bd438e601b130e243d
Jul 18 01:36:53.120340 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.120303    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerID="cd1bbf9e971d44d1ba57740df023c6a6c92f3094b52caf061cf9dfbb3e0ac946" exitCode=6
Jul 18 01:36:53.120824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.120371    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" event={"ID":"e9c37383-5e0d-4682-b4a4-a0210f7b9759","Type":"ContainerDied","Data":"cd1bbf9e971d44d1ba57740df023c6a6c92f3094b52caf061cf9dfbb3e0ac946"}
Jul 18 01:36:53.120824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.120432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" event={"ID":"e9c37383-5e0d-4682-b4a4-a0210f7b9759","Type":"ContainerStarted","Data":"4c679bee2b80fef4788c051ce91729ad3c6fe1bad9d507bd438e601b130e243d"}
Jul 18 01:36:53.120824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.120658    2567 scope.go:117] "RemoveContainer" containerID="cd1bbf9e971d44d1ba57740df023c6a6c92f3094b52caf061cf9dfbb3e0ac946"
Jul 18 01:36:53.136358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.136336    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"]
Jul 18 01:36:53.289242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.289212    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:53.392386 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.392304    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abfdf46-caf5-4e8b-a011-a3ac96893119-maas-api-service-ca\") pod \"9abfdf46-caf5-4e8b-a011-a3ac96893119\" (UID: \"9abfdf46-caf5-4e8b-a011-a3ac96893119\") "
Jul 18 01:36:53.392712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.392687    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abfdf46-caf5-4e8b-a011-a3ac96893119-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9abfdf46-caf5-4e8b-a011-a3ac96893119" (UID: "9abfdf46-caf5-4e8b-a011-a3ac96893119"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:36:53.493564 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:53.493525    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abfdf46-caf5-4e8b-a011-a3ac96893119-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:36:54.130535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.130506    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"
Jul 18 01:36:54.130965 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.130549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8" event={"ID":"9abfdf46-caf5-4e8b-a011-a3ac96893119","Type":"ContainerDied","Data":"0c07c39e5b84d6f82e13d2abd721aa51e587c6d955882ffeeaf2608ea32658c9"}
Jul 18 01:36:54.130965 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.130608    2567 scope.go:117] "RemoveContainer" containerID="229e94368039d455cabacd9ffed6ed860ca12b0b1909e18293b74347a5b42fe4"
Jul 18 01:36:54.132571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.132544    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerID="1b70704845d92e8e51956c9c4fcb1b21373e443b40ffcc0a1173aa679c66e268" exitCode=6
Jul 18 01:36:54.132697 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.132613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" event={"ID":"e9c37383-5e0d-4682-b4a4-a0210f7b9759","Type":"ContainerDied","Data":"1b70704845d92e8e51956c9c4fcb1b21373e443b40ffcc0a1173aa679c66e268"}
Jul 18 01:36:54.132835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.132819    2567 scope.go:117] "RemoveContainer" containerID="1b70704845d92e8e51956c9c4fcb1b21373e443b40ffcc0a1173aa679c66e268"
Jul 18 01:36:54.133065 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:54.133044    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq_odh-ai-gateway-infra(e9c37383-5e0d-4682-b4a4-a0210f7b9759)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759"
Jul 18 01:36:54.140343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.140323    2567 scope.go:117] "RemoveContainer" containerID="cd1bbf9e971d44d1ba57740df023c6a6c92f3094b52caf061cf9dfbb3e0ac946"
Jul 18 01:36:54.164960 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.164934    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"]
Jul 18 01:36:54.167052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.167031    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-47mv8"]
Jul 18 01:36:54.919171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:54.919134    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" path="/var/lib/kubelet/pods/9abfdf46-caf5-4e8b-a011-a3ac96893119/volumes"
Jul 18 01:36:55.140362 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:55.140325    2567 scope.go:117] "RemoveContainer" containerID="1b70704845d92e8e51956c9c4fcb1b21373e443b40ffcc0a1173aa679c66e268"
Jul 18 01:36:55.140928 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:36:55.140588    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq_odh-ai-gateway-infra(e9c37383-5e0d-4682-b4a4-a0210f7b9759)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759"
Jul 18 01:36:58.520427 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:36:58.520390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:37:00.609369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:00.609331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:37:03.913586 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:03.913557    2567 scope.go:117] "RemoveContainer" containerID="aef0d797150954a752ba26f67cacdaf54f065e499693e1f99ac70a78bd542f13"
Jul 18 01:37:04.192516 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:04.192428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" event={"ID":"05f690d2-03a1-4729-a745-0ab5b40cf28e","Type":"ContainerStarted","Data":"20721329ca79e6b61d9aac130b59236bdd50c758dd558ea60479786a50ee5819"}
Jul 18 01:37:04.208161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:04.208098    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" podStartSLOduration=18.208082211 podStartE2EDuration="18.208082211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:37:04.206234605 +0000 UTC m=+3403.733739565" watchObservedRunningTime="2026-07-18 01:37:04.208082211 +0000 UTC m=+3403.735587171"
Jul 18 01:37:05.200429 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:05.200395    2567 generic.go:356] "Generic (PLEG): container finished" podID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerID="20721329ca79e6b61d9aac130b59236bdd50c758dd558ea60479786a50ee5819" exitCode=6
Jul 18 01:37:05.200902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:05.200473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" event={"ID":"05f690d2-03a1-4729-a745-0ab5b40cf28e","Type":"ContainerDied","Data":"20721329ca79e6b61d9aac130b59236bdd50c758dd558ea60479786a50ee5819"}
Jul 18 01:37:05.200902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:05.200557    2567 scope.go:117] "RemoveContainer" containerID="aef0d797150954a752ba26f67cacdaf54f065e499693e1f99ac70a78bd542f13"
Jul 18 01:37:05.201012 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:05.200935    2567 scope.go:117] "RemoveContainer" containerID="20721329ca79e6b61d9aac130b59236bdd50c758dd558ea60479786a50ee5819"
Jul 18 01:37:05.201203 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:05.201177    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-665g5_odh-ai-gateway-infra(05f690d2-03a1-4729-a745-0ab5b40cf28e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e"
Jul 18 01:37:05.229625 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:05.229595    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"]
Jul 18 01:37:06.357905 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:06.357874    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:37:06.423616 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:06.423580    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f690d2-03a1-4729-a745-0ab5b40cf28e-maas-api-service-ca\") pod \"05f690d2-03a1-4729-a745-0ab5b40cf28e\" (UID: \"05f690d2-03a1-4729-a745-0ab5b40cf28e\") "
Jul 18 01:37:06.423939 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:06.423916    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05f690d2-03a1-4729-a745-0ab5b40cf28e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05f690d2-03a1-4729-a745-0ab5b40cf28e" (UID: "05f690d2-03a1-4729-a745-0ab5b40cf28e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:37:06.525148 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:06.525064    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f690d2-03a1-4729-a745-0ab5b40cf28e-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:06.913144 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:06.913110    2567 scope.go:117] "RemoveContainer" containerID="1b70704845d92e8e51956c9c4fcb1b21373e443b40ffcc0a1173aa679c66e268"
Jul 18 01:37:07.216718 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.216618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" event={"ID":"e9c37383-5e0d-4682-b4a4-a0210f7b9759","Type":"ContainerStarted","Data":"9f817d9d242054e0a00ca0a483d663b190211d29207631abf89d7fdbf210ae3f"}
Jul 18 01:37:07.217913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.217890    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"
Jul 18 01:37:07.217913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.217907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5" event={"ID":"05f690d2-03a1-4729-a745-0ab5b40cf28e","Type":"ContainerDied","Data":"cd1c9d2a28f17ec252435a684fb1b29403394572b8eea60498db3609a8f9729b"}
Jul 18 01:37:07.218114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.217961    2567 scope.go:117] "RemoveContainer" containerID="20721329ca79e6b61d9aac130b59236bdd50c758dd558ea60479786a50ee5819"
Jul 18 01:37:07.234520 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.234474    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" podStartSLOduration=16.234459327 podStartE2EDuration="16.234459327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:37:07.232597687 +0000 UTC m=+3406.760102646" watchObservedRunningTime="2026-07-18 01:37:07.234459327 +0000 UTC m=+3406.761964287"
Jul 18 01:37:07.245902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.245877    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"]
Jul 18 01:37:07.248052 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:07.248030    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-665g5"]
Jul 18 01:37:08.225039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:08.224999    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerID="9f817d9d242054e0a00ca0a483d663b190211d29207631abf89d7fdbf210ae3f" exitCode=6
Jul 18 01:37:08.225514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:08.225071    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" event={"ID":"e9c37383-5e0d-4682-b4a4-a0210f7b9759","Type":"ContainerDied","Data":"9f817d9d242054e0a00ca0a483d663b190211d29207631abf89d7fdbf210ae3f"}
Jul 18 01:37:08.225514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:08.225138    2567 scope.go:117] "RemoveContainer" containerID="1b70704845d92e8e51956c9c4fcb1b21373e443b40ffcc0a1173aa679c66e268"
Jul 18 01:37:08.225514 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:08.225452    2567 scope.go:117] "RemoveContainer" containerID="9f817d9d242054e0a00ca0a483d663b190211d29207631abf89d7fdbf210ae3f"
Jul 18 01:37:08.225728 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:08.225681    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq_odh-ai-gateway-infra(e9c37383-5e0d-4682-b4a4-a0210f7b9759)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759"
Jul 18 01:37:08.248485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:08.248451    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"]
Jul 18 01:37:08.919922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:08.919886    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" path="/var/lib/kubelet/pods/05f690d2-03a1-4729-a745-0ab5b40cf28e/volumes"
Jul 18 01:37:09.385368 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:09.385340    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:37:09.452329 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:09.452258    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c37383-5e0d-4682-b4a4-a0210f7b9759-maas-api-service-ca\") pod \"e9c37383-5e0d-4682-b4a4-a0210f7b9759\" (UID: \"e9c37383-5e0d-4682-b4a4-a0210f7b9759\") "
Jul 18 01:37:09.452651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:09.452623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c37383-5e0d-4682-b4a4-a0210f7b9759-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9c37383-5e0d-4682-b4a4-a0210f7b9759" (UID: "e9c37383-5e0d-4682-b4a4-a0210f7b9759"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:37:09.553248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:09.553148    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c37383-5e0d-4682-b4a4-a0210f7b9759-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:10.240866 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:10.240838    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"
Jul 18 01:37:10.241064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:10.240871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq" event={"ID":"e9c37383-5e0d-4682-b4a4-a0210f7b9759","Type":"ContainerDied","Data":"4c679bee2b80fef4788c051ce91729ad3c6fe1bad9d507bd438e601b130e243d"}
Jul 18 01:37:10.241064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:10.240931    2567 scope.go:117] "RemoveContainer" containerID="9f817d9d242054e0a00ca0a483d663b190211d29207631abf89d7fdbf210ae3f"
Jul 18 01:37:10.263223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:10.263182    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"]
Jul 18 01:37:10.275972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:10.275936    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-vhzfq"]
Jul 18 01:37:10.919559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:10.919527    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" path="/var/lib/kubelet/pods/e9c37383-5e0d-4682-b4a4-a0210f7b9759/volumes"
Jul 18 01:37:16.301350 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:16.301311    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:37:23.833176 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:23.833139    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"]
Jul 18 01:37:23.853206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:23.853170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:37:24.726156 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:24.726123    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:37:25.716247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.716198    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-140695167777712"]
Jul 18 01:37:25.717064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717035    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717067    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717083    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717092    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717112    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717123    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717151    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717160    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717178    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717186    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717198    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717206    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717225    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717233    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717256    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717281    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717473    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717483    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717498    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717508    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717518    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717532    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c37383-5e0d-4682-b4a4-a0210f7b9759" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717541    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:25.717655 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.717551    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:25.723227 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.723190    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:25.723682 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.723648    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167777712"]
Jul 18 01:37:25.725835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.725810    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fr2nf\""
Jul 18 01:37:25.820621 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.820583    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmnm\" (UniqueName: \"kubernetes.io/projected/a89e7bca-0405-4391-854f-e672364cf636-kube-api-access-njmnm\") pod \"test-curl-7531-140695167777712\" (UID: \"a89e7bca-0405-4391-854f-e672364cf636\") " pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:25.921951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.921919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmnm\" (UniqueName: \"kubernetes.io/projected/a89e7bca-0405-4391-854f-e672364cf636-kube-api-access-njmnm\") pod \"test-curl-7531-140695167777712\" (UID: \"a89e7bca-0405-4391-854f-e672364cf636\") " pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:25.929887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:25.929867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmnm\" (UniqueName: \"kubernetes.io/projected/a89e7bca-0405-4391-854f-e672364cf636-kube-api-access-njmnm\") pod \"test-curl-7531-140695167777712\" (UID: \"a89e7bca-0405-4391-854f-e672364cf636\") " pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:26.036345 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:26.036233    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:26.180795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:26.180769    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167777712"]
Jul 18 01:37:26.182288 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:26.182236    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda89e7bca_0405_4391_854f_e672364cf636.slice/crio-64e821f468d185d99dfa733b47b557dfd06b63e82b98af9cdd87d21ac439fbcb WatchSource:0}: Error finding container 64e821f468d185d99dfa733b47b557dfd06b63e82b98af9cdd87d21ac439fbcb: Status 404 returned error can't find the container with id 64e821f468d185d99dfa733b47b557dfd06b63e82b98af9cdd87d21ac439fbcb
Jul 18 01:37:26.331887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:26.331848    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167777712" event={"ID":"a89e7bca-0405-4391-854f-e672364cf636","Type":"ContainerStarted","Data":"64e821f468d185d99dfa733b47b557dfd06b63e82b98af9cdd87d21ac439fbcb"}
Jul 18 01:37:27.339197 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:27.339163    2567 generic.go:356] "Generic (PLEG): container finished" podID="a89e7bca-0405-4391-854f-e672364cf636" containerID="6690564ce0489df3b119236634196cea19202d4ad4b5ffb90222fc09b567a883" exitCode=0
Jul 18 01:37:27.339716 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:27.339249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167777712" event={"ID":"a89e7bca-0405-4391-854f-e672364cf636","Type":"ContainerDied","Data":"6690564ce0489df3b119236634196cea19202d4ad4b5ffb90222fc09b567a883"}
Jul 18 01:37:28.488117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.488087    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:28.506707 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.506678    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7531-140695167777712_a89e7bca-0405-4391-854f-e672364cf636/test-curl-7531-140695167777712/0.log"
Jul 18 01:37:28.526917 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.526886    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-140695167777712"]
Jul 18 01:37:28.528947 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.528927    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-140695167777712"]
Jul 18 01:37:28.645764 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.645677    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-140695167828144"]
Jul 18 01:37:28.646114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.646099    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89e7bca-0405-4391-854f-e672364cf636" containerName="test-curl-7531-140695167777712"
Jul 18 01:37:28.646158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.646117    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89e7bca-0405-4391-854f-e672364cf636" containerName="test-curl-7531-140695167777712"
Jul 18 01:37:28.646158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.646128    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:28.646158 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.646133    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f690d2-03a1-4729-a745-0ab5b40cf28e" containerName="revoke-keys"
Jul 18 01:37:28.646284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.646196    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abfdf46-caf5-4e8b-a011-a3ac96893119" containerName="revoke-keys"
Jul 18 01:37:28.646284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.646206    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89e7bca-0405-4391-854f-e672364cf636" containerName="test-curl-7531-140695167777712"
Jul 18 01:37:28.649208 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.649191    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:28.649450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.649433    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njmnm\" (UniqueName: \"kubernetes.io/projected/a89e7bca-0405-4391-854f-e672364cf636-kube-api-access-njmnm\") pod \"a89e7bca-0405-4391-854f-e672364cf636\" (UID: \"a89e7bca-0405-4391-854f-e672364cf636\") "
Jul 18 01:37:28.651787 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.651754    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89e7bca-0405-4391-854f-e672364cf636-kube-api-access-njmnm" (OuterVolumeSpecName: "kube-api-access-njmnm") pod "a89e7bca-0405-4391-854f-e672364cf636" (UID: "a89e7bca-0405-4391-854f-e672364cf636"). InnerVolumeSpecName "kube-api-access-njmnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:28.656026 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.656003    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167828144"]
Jul 18 01:37:28.750748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.750698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsbjm\" (UniqueName: \"kubernetes.io/projected/bd6a13f2-bb84-41bb-ab91-1ffb18567c37-kube-api-access-gsbjm\") pod \"test-curl-7531-140695167828144\" (UID: \"bd6a13f2-bb84-41bb-ab91-1ffb18567c37\") " pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:28.750936 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.750876    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njmnm\" (UniqueName: \"kubernetes.io/projected/a89e7bca-0405-4391-854f-e672364cf636-kube-api-access-njmnm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:28.851693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.851641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsbjm\" (UniqueName: \"kubernetes.io/projected/bd6a13f2-bb84-41bb-ab91-1ffb18567c37-kube-api-access-gsbjm\") pod \"test-curl-7531-140695167828144\" (UID: \"bd6a13f2-bb84-41bb-ab91-1ffb18567c37\") " pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:28.859354 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.859320    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsbjm\" (UniqueName: \"kubernetes.io/projected/bd6a13f2-bb84-41bb-ab91-1ffb18567c37-kube-api-access-gsbjm\") pod \"test-curl-7531-140695167828144\" (UID: \"bd6a13f2-bb84-41bb-ab91-1ffb18567c37\") " pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:28.920599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.920512    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89e7bca-0405-4391-854f-e672364cf636" path="/var/lib/kubelet/pods/a89e7bca-0405-4391-854f-e672364cf636/volumes"
Jul 18 01:37:28.967546 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:28.967510    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:29.107020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:29.106993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167828144"]
Jul 18 01:37:29.108875 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:29.108837    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd6a13f2_bb84_41bb_ab91_1ffb18567c37.slice/crio-378c59b591feaae3a0ebc71c788cc37ee34bb226dcb2ccbac24199852478a43e WatchSource:0}: Error finding container 378c59b591feaae3a0ebc71c788cc37ee34bb226dcb2ccbac24199852478a43e: Status 404 returned error can't find the container with id 378c59b591feaae3a0ebc71c788cc37ee34bb226dcb2ccbac24199852478a43e
Jul 18 01:37:29.352457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:29.352429    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167777712"
Jul 18 01:37:29.352457 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:29.352454    2567 scope.go:117] "RemoveContainer" containerID="6690564ce0489df3b119236634196cea19202d4ad4b5ffb90222fc09b567a883"
Jul 18 01:37:29.353968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:29.353935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167828144" event={"ID":"bd6a13f2-bb84-41bb-ab91-1ffb18567c37","Type":"ContainerStarted","Data":"378c59b591feaae3a0ebc71c788cc37ee34bb226dcb2ccbac24199852478a43e"}
Jul 18 01:37:30.360966 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:30.360928    2567 generic.go:356] "Generic (PLEG): container finished" podID="bd6a13f2-bb84-41bb-ab91-1ffb18567c37" containerID="c908a46ea90781cab2c593104f1d32fda06f486ddebc36b7526a0a6293054064" exitCode=0
Jul 18 01:37:30.361471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:30.361021    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167828144" event={"ID":"bd6a13f2-bb84-41bb-ab91-1ffb18567c37","Type":"ContainerDied","Data":"c908a46ea90781cab2c593104f1d32fda06f486ddebc36b7526a0a6293054064"}
Jul 18 01:37:31.217207 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.217168    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:37:31.510597 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.510571    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:31.530343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.530317    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7531-140695167828144_bd6a13f2-bb84-41bb-ab91-1ffb18567c37/test-curl-7531-140695167828144/0.log"
Jul 18 01:37:31.556367 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.556320    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-140695167828144"]
Jul 18 01:37:31.558984 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.558955    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-140695167828144"]
Jul 18 01:37:31.683321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.683252    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsbjm\" (UniqueName: \"kubernetes.io/projected/bd6a13f2-bb84-41bb-ab91-1ffb18567c37-kube-api-access-gsbjm\") pod \"bd6a13f2-bb84-41bb-ab91-1ffb18567c37\" (UID: \"bd6a13f2-bb84-41bb-ab91-1ffb18567c37\") "
Jul 18 01:37:31.685599 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.685561    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6a13f2-bb84-41bb-ab91-1ffb18567c37-kube-api-access-gsbjm" (OuterVolumeSpecName: "kube-api-access-gsbjm") pod "bd6a13f2-bb84-41bb-ab91-1ffb18567c37" (UID: "bd6a13f2-bb84-41bb-ab91-1ffb18567c37"). InnerVolumeSpecName "kube-api-access-gsbjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:31.728364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.728329    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:31.728713 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.728700    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd6a13f2-bb84-41bb-ab91-1ffb18567c37" containerName="test-curl-7531-140695167828144"
Jul 18 01:37:31.728759 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.728714    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6a13f2-bb84-41bb-ab91-1ffb18567c37" containerName="test-curl-7531-140695167828144"
Jul 18 01:37:31.728803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.728794    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd6a13f2-bb84-41bb-ab91-1ffb18567c37" containerName="test-curl-7531-140695167828144"
Jul 18 01:37:31.731800 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.731782    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:31.739432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.739411    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:31.784476 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.784432    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsbjm\" (UniqueName: \"kubernetes.io/projected/bd6a13f2-bb84-41bb-ab91-1ffb18567c37-kube-api-access-gsbjm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:31.884893 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.884864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzgp9\" (UniqueName: \"kubernetes.io/projected/71716113-909d-4f47-9b6d-81ce12b91b28-kube-api-access-kzgp9\") pod \"test-curl-7531-140695167939904\" (UID: \"71716113-909d-4f47-9b6d-81ce12b91b28\") " pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:31.986433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.986389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzgp9\" (UniqueName: \"kubernetes.io/projected/71716113-909d-4f47-9b6d-81ce12b91b28-kube-api-access-kzgp9\") pod \"test-curl-7531-140695167939904\" (UID: \"71716113-909d-4f47-9b6d-81ce12b91b28\") " pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:31.994651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:31.994561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzgp9\" (UniqueName: \"kubernetes.io/projected/71716113-909d-4f47-9b6d-81ce12b91b28-kube-api-access-kzgp9\") pod \"test-curl-7531-140695167939904\" (UID: \"71716113-909d-4f47-9b6d-81ce12b91b28\") " pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:32.042439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:32.042402    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:32.193889 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:32.193864    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:32.196469 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:32.196438    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71716113_909d_4f47_9b6d_81ce12b91b28.slice/crio-92c6788715f5ebc2bf13e337da6c267dfb7cfc5b450f91fb661f4e20f808171d WatchSource:0}: Error finding container 92c6788715f5ebc2bf13e337da6c267dfb7cfc5b450f91fb661f4e20f808171d: Status 404 returned error can't find the container with id 92c6788715f5ebc2bf13e337da6c267dfb7cfc5b450f91fb661f4e20f808171d
Jul 18 01:37:32.373695 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:32.373660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167939904" event={"ID":"71716113-909d-4f47-9b6d-81ce12b91b28","Type":"ContainerStarted","Data":"92c6788715f5ebc2bf13e337da6c267dfb7cfc5b450f91fb661f4e20f808171d"}
Jul 18 01:37:32.375013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:32.374990    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="378c59b591feaae3a0ebc71c788cc37ee34bb226dcb2ccbac24199852478a43e"
Jul 18 01:37:32.375013 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:32.374998    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:32.385627 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:32.385598    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-140695167828144\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="bd6a13f2-bb84-41bb-ab91-1ffb18567c37" pod="opendatahub/test-curl-7531-140695167828144"
Jul 18 01:37:32.920074 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:32.920041    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd6a13f2-bb84-41bb-ab91-1ffb18567c37" path="/var/lib/kubelet/pods/bd6a13f2-bb84-41bb-ab91-1ffb18567c37/volumes"
Jul 18 01:37:33.381515 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:33.381482    2567 generic.go:356] "Generic (PLEG): container finished" podID="71716113-909d-4f47-9b6d-81ce12b91b28" containerID="a1043e5d981b18e985e5f4992b6d876382e376fbceb3f5866650b1fb12ab71a7" exitCode=0
Jul 18 01:37:33.381725 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:33.381532    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167939904" event={"ID":"71716113-909d-4f47-9b6d-81ce12b91b28","Type":"ContainerDied","Data":"a1043e5d981b18e985e5f4992b6d876382e376fbceb3f5866650b1fb12ab71a7"}
Jul 18 01:37:34.534053 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.534017    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:34.552559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.552528    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7531-140695167939904_71716113-909d-4f47-9b6d-81ce12b91b28/test-curl-7531-140695167939904/0.log"
Jul 18 01:37:34.578159 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.578123    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:34.581955 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.581921    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:34.702566 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.702480    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:34.702942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.702927    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71716113-909d-4f47-9b6d-81ce12b91b28" containerName="test-curl-7531-140695167939904"
Jul 18 01:37:34.702993 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.702947    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="71716113-909d-4f47-9b6d-81ce12b91b28" containerName="test-curl-7531-140695167939904"
Jul 18 01:37:34.703033 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.703018    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="71716113-909d-4f47-9b6d-81ce12b91b28" containerName="test-curl-7531-140695167939904"
Jul 18 01:37:34.706068 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.706044    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:34.709527 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.709504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:34.714171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.714034    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzgp9\" (UniqueName: \"kubernetes.io/projected/71716113-909d-4f47-9b6d-81ce12b91b28-kube-api-access-kzgp9\") pod \"71716113-909d-4f47-9b6d-81ce12b91b28\" (UID: \"71716113-909d-4f47-9b6d-81ce12b91b28\") "
Jul 18 01:37:34.716394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.716364    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71716113-909d-4f47-9b6d-81ce12b91b28-kube-api-access-kzgp9" (OuterVolumeSpecName: "kube-api-access-kzgp9") pod "71716113-909d-4f47-9b6d-81ce12b91b28" (UID: "71716113-909d-4f47-9b6d-81ce12b91b28"). InnerVolumeSpecName "kube-api-access-kzgp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:34.815535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.815491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8slxw\" (UniqueName: \"kubernetes.io/projected/21f32f05-c3f4-45a2-bfd5-863f2fc00e91-kube-api-access-8slxw\") pod \"test-curl-7531-140695167939904\" (UID: \"21f32f05-c3f4-45a2-bfd5-863f2fc00e91\") " pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:34.815755 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.815688    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzgp9\" (UniqueName: \"kubernetes.io/projected/71716113-909d-4f47-9b6d-81ce12b91b28-kube-api-access-kzgp9\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:34.916974 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.916939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8slxw\" (UniqueName: \"kubernetes.io/projected/21f32f05-c3f4-45a2-bfd5-863f2fc00e91-kube-api-access-8slxw\") pod \"test-curl-7531-140695167939904\" (UID: \"21f32f05-c3f4-45a2-bfd5-863f2fc00e91\") " pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:34.919523 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.919498    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71716113-909d-4f47-9b6d-81ce12b91b28" path="/var/lib/kubelet/pods/71716113-909d-4f47-9b6d-81ce12b91b28/volumes"
Jul 18 01:37:34.924955 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:34.924922    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8slxw\" (UniqueName: \"kubernetes.io/projected/21f32f05-c3f4-45a2-bfd5-863f2fc00e91-kube-api-access-8slxw\") pod \"test-curl-7531-140695167939904\" (UID: \"21f32f05-c3f4-45a2-bfd5-863f2fc00e91\") " pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:35.017236 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:35.017139    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:35.176552 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:35.176526    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:35.178336 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:35.178309    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21f32f05_c3f4_45a2_bfd5_863f2fc00e91.slice/crio-8df56f9052f27f66332175bbfc563ecf03266c4b964c2cf76cfe6ca317450f93 WatchSource:0}: Error finding container 8df56f9052f27f66332175bbfc563ecf03266c4b964c2cf76cfe6ca317450f93: Status 404 returned error can't find the container with id 8df56f9052f27f66332175bbfc563ecf03266c4b964c2cf76cfe6ca317450f93
Jul 18 01:37:35.394689 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:35.394651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167939904" event={"ID":"21f32f05-c3f4-45a2-bfd5-863f2fc00e91","Type":"ContainerStarted","Data":"8df56f9052f27f66332175bbfc563ecf03266c4b964c2cf76cfe6ca317450f93"}
Jul 18 01:37:35.396321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:35.396302    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:35.396321 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:35.396313    2567 scope.go:117] "RemoveContainer" containerID="a1043e5d981b18e985e5f4992b6d876382e376fbceb3f5866650b1fb12ab71a7"
Jul 18 01:37:36.404819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:36.404780    2567 generic.go:356] "Generic (PLEG): container finished" podID="21f32f05-c3f4-45a2-bfd5-863f2fc00e91" containerID="aad5812cc790344b4db9ae45eaa1017d8dda5e646a195b34e2d651baf941c820" exitCode=0
Jul 18 01:37:36.405285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:36.404834    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167939904" event={"ID":"21f32f05-c3f4-45a2-bfd5-863f2fc00e91","Type":"ContainerDied","Data":"aad5812cc790344b4db9ae45eaa1017d8dda5e646a195b34e2d651baf941c820"}
Jul 18 01:37:37.568388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.568362    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:37.586207 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.586180    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7531-140695167939904_21f32f05-c3f4-45a2-bfd5-863f2fc00e91/test-curl-7531-140695167939904/0.log"
Jul 18 01:37:37.607933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.607895    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:37.612921 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.612894    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-140695167939904"]
Jul 18 01:37:37.642428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.642394    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8slxw\" (UniqueName: \"kubernetes.io/projected/21f32f05-c3f4-45a2-bfd5-863f2fc00e91-kube-api-access-8slxw\") pod \"21f32f05-c3f4-45a2-bfd5-863f2fc00e91\" (UID: \"21f32f05-c3f4-45a2-bfd5-863f2fc00e91\") "
Jul 18 01:37:37.644761 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.644732    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f32f05-c3f4-45a2-bfd5-863f2fc00e91-kube-api-access-8slxw" (OuterVolumeSpecName: "kube-api-access-8slxw") pod "21f32f05-c3f4-45a2-bfd5-863f2fc00e91" (UID: "21f32f05-c3f4-45a2-bfd5-863f2fc00e91"). InnerVolumeSpecName "kube-api-access-8slxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:37.736568 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.736477    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:37.736882 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.736867    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f32f05-c3f4-45a2-bfd5-863f2fc00e91" containerName="test-curl-7531-140695167939904"
Jul 18 01:37:37.736882 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.736882    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f32f05-c3f4-45a2-bfd5-863f2fc00e91" containerName="test-curl-7531-140695167939904"
Jul 18 01:37:37.736975 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.736954    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f32f05-c3f4-45a2-bfd5-863f2fc00e91" containerName="test-curl-7531-140695167939904"
Jul 18 01:37:37.740029 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.740012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:37.743073 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.743048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xn6\" (UniqueName: \"kubernetes.io/projected/b4b4f51b-d779-4441-b209-ef5a3a318dbd-kube-api-access-x5xn6\") pod \"test-curl-7531-140695167940912\" (UID: \"b4b4f51b-d779-4441-b209-ef5a3a318dbd\") " pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:37.743218 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.743094    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8slxw\" (UniqueName: \"kubernetes.io/projected/21f32f05-c3f4-45a2-bfd5-863f2fc00e91-kube-api-access-8slxw\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:37.748681 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.748658    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:37.843660 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.843614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xn6\" (UniqueName: \"kubernetes.io/projected/b4b4f51b-d779-4441-b209-ef5a3a318dbd-kube-api-access-x5xn6\") pod \"test-curl-7531-140695167940912\" (UID: \"b4b4f51b-d779-4441-b209-ef5a3a318dbd\") " pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:37.852357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:37.852329    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xn6\" (UniqueName: \"kubernetes.io/projected/b4b4f51b-d779-4441-b209-ef5a3a318dbd-kube-api-access-x5xn6\") pod \"test-curl-7531-140695167940912\" (UID: \"b4b4f51b-d779-4441-b209-ef5a3a318dbd\") " pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:38.050539 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.050431    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:38.193654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.193625    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:38.196374 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:38.196332    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4b4f51b_d779_4441_b209_ef5a3a318dbd.slice/crio-eb8e84bcbfe6b416e7fe890a7fe0fc59f7ffba2ef222561ce2e92ed61cc13dc3 WatchSource:0}: Error finding container eb8e84bcbfe6b416e7fe890a7fe0fc59f7ffba2ef222561ce2e92ed61cc13dc3: Status 404 returned error can't find the container with id eb8e84bcbfe6b416e7fe890a7fe0fc59f7ffba2ef222561ce2e92ed61cc13dc3
Jul 18 01:37:38.417161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.417131    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:38.417405 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.417136    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8df56f9052f27f66332175bbfc563ecf03266c4b964c2cf76cfe6ca317450f93"
Jul 18 01:37:38.418416 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.418391    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167940912" event={"ID":"b4b4f51b-d779-4441-b209-ef5a3a318dbd","Type":"ContainerStarted","Data":"eb8e84bcbfe6b416e7fe890a7fe0fc59f7ffba2ef222561ce2e92ed61cc13dc3"}
Jul 18 01:37:38.427377 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:38.427351    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-140695167939904\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="21f32f05-c3f4-45a2-bfd5-863f2fc00e91" pod="opendatahub/test-curl-7531-140695167939904"
Jul 18 01:37:38.856473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.856423    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd" podUID="29e144c0-acb4-43dc-9d6a-db02c21944ff" containerName="main" containerID="cri-o://6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18" gracePeriod=45
Jul 18 01:37:38.921002 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:38.920968    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f32f05-c3f4-45a2-bfd5-863f2fc00e91" path="/var/lib/kubelet/pods/21f32f05-c3f4-45a2-bfd5-863f2fc00e91/volumes"
Jul 18 01:37:39.123991 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.123957    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:37:39.156883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.156847    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-model-cache\") pod \"29e144c0-acb4-43dc-9d6a-db02c21944ff\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") "
Jul 18 01:37:39.157096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.156903    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv4k7\" (UniqueName: \"kubernetes.io/projected/29e144c0-acb4-43dc-9d6a-db02c21944ff-kube-api-access-sv4k7\") pod \"29e144c0-acb4-43dc-9d6a-db02c21944ff\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") "
Jul 18 01:37:39.157096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.156946    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-dshm\") pod \"29e144c0-acb4-43dc-9d6a-db02c21944ff\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") "
Jul 18 01:37:39.157096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.157002    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-tmp-dir\") pod \"29e144c0-acb4-43dc-9d6a-db02c21944ff\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") "
Jul 18 01:37:39.157096 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.157043    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e144c0-acb4-43dc-9d6a-db02c21944ff-tls-certs\") pod \"29e144c0-acb4-43dc-9d6a-db02c21944ff\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") "
Jul 18 01:37:39.157358 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.157187    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "29e144c0-acb4-43dc-9d6a-db02c21944ff" (UID: "29e144c0-acb4-43dc-9d6a-db02c21944ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:37:39.157426 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.157396    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29e144c0-acb4-43dc-9d6a-db02c21944ff" (UID: "29e144c0-acb4-43dc-9d6a-db02c21944ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:37:39.157565 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.157545    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-tmp-dir\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:39.157626 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.157573    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-model-cache\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:39.159911 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.159880    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-dshm" (OuterVolumeSpecName: "dshm") pod "29e144c0-acb4-43dc-9d6a-db02c21944ff" (UID: "29e144c0-acb4-43dc-9d6a-db02c21944ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:37:39.160085 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.160067    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e144c0-acb4-43dc-9d6a-db02c21944ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29e144c0-acb4-43dc-9d6a-db02c21944ff" (UID: "29e144c0-acb4-43dc-9d6a-db02c21944ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:37:39.160153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.160084    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e144c0-acb4-43dc-9d6a-db02c21944ff-kube-api-access-sv4k7" (OuterVolumeSpecName: "kube-api-access-sv4k7") pod "29e144c0-acb4-43dc-9d6a-db02c21944ff" (UID: "29e144c0-acb4-43dc-9d6a-db02c21944ff"). InnerVolumeSpecName "kube-api-access-sv4k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:39.258769 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.258733    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-home\") pod \"29e144c0-acb4-43dc-9d6a-db02c21944ff\" (UID: \"29e144c0-acb4-43dc-9d6a-db02c21944ff\") "
Jul 18 01:37:39.258961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.258927    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv4k7\" (UniqueName: \"kubernetes.io/projected/29e144c0-acb4-43dc-9d6a-db02c21944ff-kube-api-access-sv4k7\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:39.258961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.258939    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-dshm\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:39.258961 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.258950    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29e144c0-acb4-43dc-9d6a-db02c21944ff-tls-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:39.259076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.259018    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-home" (OuterVolumeSpecName: "home") pod "29e144c0-acb4-43dc-9d6a-db02c21944ff" (UID: "29e144c0-acb4-43dc-9d6a-db02c21944ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:37:39.360454 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.360368    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29e144c0-acb4-43dc-9d6a-db02c21944ff-home\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:39.426870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.426835    2567 generic.go:356] "Generic (PLEG): container finished" podID="29e144c0-acb4-43dc-9d6a-db02c21944ff" containerID="6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18" exitCode=0
Jul 18 01:37:39.427077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.426878    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"
Jul 18 01:37:39.427077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.426921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd" event={"ID":"29e144c0-acb4-43dc-9d6a-db02c21944ff","Type":"ContainerDied","Data":"6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18"}
Jul 18 01:37:39.427077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.426975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd" event={"ID":"29e144c0-acb4-43dc-9d6a-db02c21944ff","Type":"ContainerDied","Data":"65d39559135b3f331f6d6bdad781738721d085905ff0af46bc8ef1bb45361d7f"}
Jul 18 01:37:39.427077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.427005    2567 scope.go:117] "RemoveContainer" containerID="6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18"
Jul 18 01:37:39.428675 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.428646    2567 generic.go:356] "Generic (PLEG): container finished" podID="b4b4f51b-d779-4441-b209-ef5a3a318dbd" containerID="58a52718ae0f095cd8fab318f4fe246d65b68c6bc032db05e8a8db9ac9a27a08" exitCode=0
Jul 18 01:37:39.428793 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.428728    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167940912" event={"ID":"b4b4f51b-d779-4441-b209-ef5a3a318dbd","Type":"ContainerDied","Data":"58a52718ae0f095cd8fab318f4fe246d65b68c6bc032db05e8a8db9ac9a27a08"}
Jul 18 01:37:39.436809 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.436792    2567 scope.go:117] "RemoveContainer" containerID="6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18"
Jul 18 01:37:39.437078 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:39.437059    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18\": container with ID starting with 6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18 not found: ID does not exist" containerID="6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18"
Jul 18 01:37:39.437188 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.437092    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18"} err="failed to get container status \"6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18\": rpc error: code = NotFound desc = could not find container \"6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18\": container with ID starting with 6ae22f8d2699978e580198b7cc56f4bc614a49a897b9ef5ecd66fc2dba52ba18 not found: ID does not exist"
Jul 18 01:37:39.454260 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.454226    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"]
Jul 18 01:37:39.457692 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:39.457664    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6b3233/rate-test-model-6b3233-kserve-85747f7d75-6gsxd"]
Jul 18 01:37:40.577594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.577568    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:40.596972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.596947    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7531-140695167940912_b4b4f51b-d779-4441-b209-ef5a3a318dbd/test-curl-7531-140695167940912/0.log"
Jul 18 01:37:40.620094 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.620058    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:40.622948 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.622919    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:40.672844 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.672806    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5xn6\" (UniqueName: \"kubernetes.io/projected/b4b4f51b-d779-4441-b209-ef5a3a318dbd-kube-api-access-x5xn6\") pod \"b4b4f51b-d779-4441-b209-ef5a3a318dbd\" (UID: \"b4b4f51b-d779-4441-b209-ef5a3a318dbd\") "
Jul 18 01:37:40.675149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.675109    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b4f51b-d779-4441-b209-ef5a3a318dbd-kube-api-access-x5xn6" (OuterVolumeSpecName: "kube-api-access-x5xn6") pod "b4b4f51b-d779-4441-b209-ef5a3a318dbd" (UID: "b4b4f51b-d779-4441-b209-ef5a3a318dbd"). InnerVolumeSpecName "kube-api-access-x5xn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:40.740313 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740214    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:40.740652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740638    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29e144c0-acb4-43dc-9d6a-db02c21944ff" containerName="main"
Jul 18 01:37:40.740696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740654    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="29e144c0-acb4-43dc-9d6a-db02c21944ff" containerName="main"
Jul 18 01:37:40.740696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740664    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4b4f51b-d779-4441-b209-ef5a3a318dbd" containerName="test-curl-7531-140695167940912"
Jul 18 01:37:40.740696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740669    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4b4f51b-d779-4441-b209-ef5a3a318dbd" containerName="test-curl-7531-140695167940912"
Jul 18 01:37:40.740804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740734    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="29e144c0-acb4-43dc-9d6a-db02c21944ff" containerName="main"
Jul 18 01:37:40.740804 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.740743    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4b4f51b-d779-4441-b209-ef5a3a318dbd" containerName="test-curl-7531-140695167940912"
Jul 18 01:37:40.743785 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.743768    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:40.748513 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.748489    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:40.774484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.774452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn5p2\" (UniqueName: \"kubernetes.io/projected/3bf43760-975b-406f-8cde-e43917388882-kube-api-access-bn5p2\") pod \"test-curl-7531-140695167940912\" (UID: \"3bf43760-975b-406f-8cde-e43917388882\") " pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:40.774640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.774604    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5xn6\" (UniqueName: \"kubernetes.io/projected/b4b4f51b-d779-4441-b209-ef5a3a318dbd-kube-api-access-x5xn6\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:40.875258 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.875228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn5p2\" (UniqueName: \"kubernetes.io/projected/3bf43760-975b-406f-8cde-e43917388882-kube-api-access-bn5p2\") pod \"test-curl-7531-140695167940912\" (UID: \"3bf43760-975b-406f-8cde-e43917388882\") " pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:40.882723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.882696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn5p2\" (UniqueName: \"kubernetes.io/projected/3bf43760-975b-406f-8cde-e43917388882-kube-api-access-bn5p2\") pod \"test-curl-7531-140695167940912\" (UID: \"3bf43760-975b-406f-8cde-e43917388882\") " pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:40.919589 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.919552    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29e144c0-acb4-43dc-9d6a-db02c21944ff" path="/var/lib/kubelet/pods/29e144c0-acb4-43dc-9d6a-db02c21944ff/volumes"
Jul 18 01:37:40.919908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:40.919895    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4b4f51b-d779-4441-b209-ef5a3a318dbd" path="/var/lib/kubelet/pods/b4b4f51b-d779-4441-b209-ef5a3a318dbd/volumes"
Jul 18 01:37:41.055760 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:41.055669    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:41.197686 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:41.197664    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:41.199774 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:41.199744    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bf43760_975b_406f_8cde_e43917388882.slice/crio-27f1e19110cec4af09037198c8e523a748519a6da81c2289c6b9fd46be7b601d WatchSource:0}: Error finding container 27f1e19110cec4af09037198c8e523a748519a6da81c2289c6b9fd46be7b601d: Status 404 returned error can't find the container with id 27f1e19110cec4af09037198c8e523a748519a6da81c2289c6b9fd46be7b601d
Jul 18 01:37:41.442343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:41.442296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167940912" event={"ID":"3bf43760-975b-406f-8cde-e43917388882","Type":"ContainerStarted","Data":"27f1e19110cec4af09037198c8e523a748519a6da81c2289c6b9fd46be7b601d"}
Jul 18 01:37:41.443470 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:41.443448    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:41.443611 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:41.443495    2567 scope.go:117] "RemoveContainer" containerID="58a52718ae0f095cd8fab318f4fe246d65b68c6bc032db05e8a8db9ac9a27a08"
Jul 18 01:37:42.451282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:42.451223    2567 generic.go:356] "Generic (PLEG): container finished" podID="3bf43760-975b-406f-8cde-e43917388882" containerID="4ddd9cce08bcc4a8a235af99236717db7c1a9723af63fe033b6c89f0371cf079" exitCode=0
Jul 18 01:37:42.451772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:42.451309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-140695167940912" event={"ID":"3bf43760-975b-406f-8cde-e43917388882","Type":"ContainerDied","Data":"4ddd9cce08bcc4a8a235af99236717db7c1a9723af63fe033b6c89f0371cf079"}
Jul 18 01:37:43.602433 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.602405    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:43.620892 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.620863    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7531-140695167940912_3bf43760-975b-406f-8cde-e43917388882/test-curl-7531-140695167940912/0.log"
Jul 18 01:37:43.642357 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.642319    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:43.646154 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.646125    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-140695167940912"]
Jul 18 01:37:43.700223 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.700189    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn5p2\" (UniqueName: \"kubernetes.io/projected/3bf43760-975b-406f-8cde-e43917388882-kube-api-access-bn5p2\") pod \"3bf43760-975b-406f-8cde-e43917388882\" (UID: \"3bf43760-975b-406f-8cde-e43917388882\") "
Jul 18 01:37:43.702441 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.702408    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf43760-975b-406f-8cde-e43917388882-kube-api-access-bn5p2" (OuterVolumeSpecName: "kube-api-access-bn5p2") pod "3bf43760-975b-406f-8cde-e43917388882" (UID: "3bf43760-975b-406f-8cde-e43917388882"). InnerVolumeSpecName "kube-api-access-bn5p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:37:43.801491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.801384    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn5p2\" (UniqueName: \"kubernetes.io/projected/3bf43760-975b-406f-8cde-e43917388882-kube-api-access-bn5p2\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:43.822934 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.822896    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"]
Jul 18 01:37:43.823368 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.823353    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf43760-975b-406f-8cde-e43917388882" containerName="test-curl-7531-140695167940912"
Jul 18 01:37:43.823412 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.823370    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf43760-975b-406f-8cde-e43917388882" containerName="test-curl-7531-140695167940912"
Jul 18 01:37:43.823465 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.823439    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf43760-975b-406f-8cde-e43917388882" containerName="test-curl-7531-140695167940912"
Jul 18 01:37:43.826544 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.826527    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:43.828829 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.828806    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:37:43.828933 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.828818    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:37:43.833509 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.833485    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"]
Jul 18 01:37:43.902299 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:43.902249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b9783e0-2647-4e89-af7e-04f2215e48d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5\" (UID: \"2b9783e0-2647-4e89-af7e-04f2215e48d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:44.002951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.002905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b9783e0-2647-4e89-af7e-04f2215e48d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5\" (UID: \"2b9783e0-2647-4e89-af7e-04f2215e48d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:44.003601 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.003582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b9783e0-2647-4e89-af7e-04f2215e48d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5\" (UID: \"2b9783e0-2647-4e89-af7e-04f2215e48d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:44.139522 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.139472    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:44.310593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.310563    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"]
Jul 18 01:37:44.312477 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:44.312453    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9783e0_2647_4e89_af7e_04f2215e48d8.slice/crio-ce18dd5d3592819a82e2a109c980a5c899640ce9c02fa2e46929e6b065396af9 WatchSource:0}: Error finding container ce18dd5d3592819a82e2a109c980a5c899640ce9c02fa2e46929e6b065396af9: Status 404 returned error can't find the container with id ce18dd5d3592819a82e2a109c980a5c899640ce9c02fa2e46929e6b065396af9
Jul 18 01:37:44.376244 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.376213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:37:44.465202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.465123    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:44.465202 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.465157    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27f1e19110cec4af09037198c8e523a748519a6da81c2289c6b9fd46be7b601d"
Jul 18 01:37:44.466541 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.466519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5" event={"ID":"2b9783e0-2647-4e89-af7e-04f2215e48d8","Type":"ContainerStarted","Data":"ce18dd5d3592819a82e2a109c980a5c899640ce9c02fa2e46929e6b065396af9"}
Jul 18 01:37:44.475660 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:44.475624    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-140695167940912\" is forbidden: User \"system:node:ip-10-0-135-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-55.ec2.internal' and this object" podUID="3bf43760-975b-406f-8cde-e43917388882" pod="opendatahub/test-curl-7531-140695167940912"
Jul 18 01:37:44.476574 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.476549    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:37:44.918824 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:44.918783    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bf43760-975b-406f-8cde-e43917388882" path="/var/lib/kubelet/pods/3bf43760-975b-406f-8cde-e43917388882/volumes"
Jul 18 01:37:45.474285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:45.474244    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b9783e0-2647-4e89-af7e-04f2215e48d8" containerID="4faa5e2b144d4b0eda4662a30cba663c374c425c3aa1bcf605345df5bdd67775" exitCode=0
Jul 18 01:37:45.474456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:45.474334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5" event={"ID":"2b9783e0-2647-4e89-af7e-04f2215e48d8","Type":"ContainerDied","Data":"4faa5e2b144d4b0eda4662a30cba663c374c425c3aa1bcf605345df5bdd67775"}
Jul 18 01:37:46.628620 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.628593    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:46.731719 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.731620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b9783e0-2647-4e89-af7e-04f2215e48d8-maas-api-service-ca\") pod \"2b9783e0-2647-4e89-af7e-04f2215e48d8\" (UID: \"2b9783e0-2647-4e89-af7e-04f2215e48d8\") "
Jul 18 01:37:46.731979 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.731955    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b9783e0-2647-4e89-af7e-04f2215e48d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b9783e0-2647-4e89-af7e-04f2215e48d8" (UID: "2b9783e0-2647-4e89-af7e-04f2215e48d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:37:46.771282 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.771227    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"]
Jul 18 01:37:46.771651 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.771638    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b9783e0-2647-4e89-af7e-04f2215e48d8" containerName="revoke-keys"
Jul 18 01:37:46.771696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.771653    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b9783e0-2647-4e89-af7e-04f2215e48d8" containerName="revoke-keys"
Jul 18 01:37:46.771740 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.771731    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b9783e0-2647-4e89-af7e-04f2215e48d8" containerName="revoke-keys"
Jul 18 01:37:46.774793 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.774775    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:37:46.781627 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.781581    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"]
Jul 18 01:37:46.832893 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.832855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4415239-3f29-4c90-9c76-a2ab4a22274d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k\" (UID: \"a4415239-3f29-4c90-9c76-a2ab4a22274d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:37:46.833064 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.832937    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b9783e0-2647-4e89-af7e-04f2215e48d8-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:37:46.934328 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.934299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4415239-3f29-4c90-9c76-a2ab4a22274d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k\" (UID: \"a4415239-3f29-4c90-9c76-a2ab4a22274d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:37:46.934892 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:46.934874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4415239-3f29-4c90-9c76-a2ab4a22274d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k\" (UID: \"a4415239-3f29-4c90-9c76-a2ab4a22274d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:37:47.088027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:47.087991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:37:47.230874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:47.230840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"]
Jul 18 01:37:47.233421 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:47.233388    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4415239_3f29_4c90_9c76_a2ab4a22274d.slice/crio-222c65bd119fdecca3863cdfb20c65bb3df09065c0e69ca1d1bbbb4b182cc724 WatchSource:0}: Error finding container 222c65bd119fdecca3863cdfb20c65bb3df09065c0e69ca1d1bbbb4b182cc724: Status 404 returned error can't find the container with id 222c65bd119fdecca3863cdfb20c65bb3df09065c0e69ca1d1bbbb4b182cc724
Jul 18 01:37:47.488161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:47.488130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" event={"ID":"a4415239-3f29-4c90-9c76-a2ab4a22274d","Type":"ContainerStarted","Data":"222c65bd119fdecca3863cdfb20c65bb3df09065c0e69ca1d1bbbb4b182cc724"}
Jul 18 01:37:47.489475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:47.489449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5" event={"ID":"2b9783e0-2647-4e89-af7e-04f2215e48d8","Type":"ContainerDied","Data":"ce18dd5d3592819a82e2a109c980a5c899640ce9c02fa2e46929e6b065396af9"}
Jul 18 01:37:47.489475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:47.489473    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5"
Jul 18 01:37:47.489637 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:47.489497    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce18dd5d3592819a82e2a109c980a5c899640ce9c02fa2e46929e6b065396af9"
Jul 18 01:37:48.496029 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:48.495998    2567 generic.go:356] "Generic (PLEG): container finished" podID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerID="77d13dd3e8014bf4b39772088a9eb8b5cdfc0b4f27633d42bc43e14dbc6da217" exitCode=6
Jul 18 01:37:48.496483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:48.496090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" event={"ID":"a4415239-3f29-4c90-9c76-a2ab4a22274d","Type":"ContainerDied","Data":"77d13dd3e8014bf4b39772088a9eb8b5cdfc0b4f27633d42bc43e14dbc6da217"}
Jul 18 01:37:48.496483 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:48.496239    2567 scope.go:117] "RemoveContainer" containerID="77d13dd3e8014bf4b39772088a9eb8b5cdfc0b4f27633d42bc43e14dbc6da217"
Jul 18 01:37:49.502954 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.502925    2567 generic.go:356] "Generic (PLEG): container finished" podID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerID="5b4c44c621aff3a19bf78f6d2bfea650e9a9a6c87c6451e2da685f85747e278a" exitCode=6
Jul 18 01:37:49.503428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.503012    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" event={"ID":"a4415239-3f29-4c90-9c76-a2ab4a22274d","Type":"ContainerDied","Data":"5b4c44c621aff3a19bf78f6d2bfea650e9a9a6c87c6451e2da685f85747e278a"}
Jul 18 01:37:49.503428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.503080    2567 scope.go:117] "RemoveContainer" containerID="77d13dd3e8014bf4b39772088a9eb8b5cdfc0b4f27633d42bc43e14dbc6da217"
Jul 18 01:37:49.503428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.503300    2567 scope.go:117] "RemoveContainer" containerID="5b4c44c621aff3a19bf78f6d2bfea650e9a9a6c87c6451e2da685f85747e278a"
Jul 18 01:37:49.503588 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:49.503534    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k_odh-ai-gateway-infra(a4415239-3f29-4c90-9c76-a2ab4a22274d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d"
Jul 18 01:37:49.534575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.534540    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"]
Jul 18 01:37:49.539332 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.539308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:37:49.544066 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.544033    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"]
Jul 18 01:37:49.659752 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.659718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168fcfa1-6846-4be8-801b-b5c583f34c01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd\" (UID: \"168fcfa1-6846-4be8-801b-b5c583f34c01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:37:49.760360 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.760253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168fcfa1-6846-4be8-801b-b5c583f34c01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd\" (UID: \"168fcfa1-6846-4be8-801b-b5c583f34c01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:37:49.760903 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.760884    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168fcfa1-6846-4be8-801b-b5c583f34c01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd\" (UID: \"168fcfa1-6846-4be8-801b-b5c583f34c01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:37:49.851553 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.851515    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:37:49.990445 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:49.990417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"]
Jul 18 01:37:49.992500 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:37:49.992469    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168fcfa1_6846_4be8_801b_b5c583f34c01.slice/crio-acbf02d4bc490bd4ed06012a2234d6fa106f9335e68058c6e4471fadaa097bbf WatchSource:0}: Error finding container acbf02d4bc490bd4ed06012a2234d6fa106f9335e68058c6e4471fadaa097bbf: Status 404 returned error can't find the container with id acbf02d4bc490bd4ed06012a2234d6fa106f9335e68058c6e4471fadaa097bbf
Jul 18 01:37:50.510631 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:50.510605    2567 scope.go:117] "RemoveContainer" containerID="5b4c44c621aff3a19bf78f6d2bfea650e9a9a6c87c6451e2da685f85747e278a"
Jul 18 01:37:50.511116 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:50.510853    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k_odh-ai-gateway-infra(a4415239-3f29-4c90-9c76-a2ab4a22274d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d"
Jul 18 01:37:50.511979 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:50.511955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerStarted","Data":"fd9904d8b48eb7de4b5604571bb6f138219257e85eed6f7e8b96d326071d7f15"}
Jul 18 01:37:50.512101 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:50.512006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerStarted","Data":"acbf02d4bc490bd4ed06012a2234d6fa106f9335e68058c6e4471fadaa097bbf"}
Jul 18 01:37:50.537845 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:50.537797    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" podStartSLOduration=1.5377796080000001 podStartE2EDuration="1.537779608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:37:50.535630946 +0000 UTC m=+3450.063135906" watchObservedRunningTime="2026-07-18 01:37:50.537779608 +0000 UTC m=+3450.065284546"
Jul 18 01:37:51.519712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:51.519672    2567 generic.go:356] "Generic (PLEG): container finished" podID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerID="fd9904d8b48eb7de4b5604571bb6f138219257e85eed6f7e8b96d326071d7f15" exitCode=6
Jul 18 01:37:51.520216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:51.519733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerDied","Data":"fd9904d8b48eb7de4b5604571bb6f138219257e85eed6f7e8b96d326071d7f15"}
Jul 18 01:37:51.520216 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:51.520037    2567 scope.go:117] "RemoveContainer" containerID="fd9904d8b48eb7de4b5604571bb6f138219257e85eed6f7e8b96d326071d7f15"
Jul 18 01:37:52.527238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:52.527138    2567 generic.go:356] "Generic (PLEG): container finished" podID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerID="e852493ad57a94a98d74008eb6fc0caf331d1d6f2bfc979cfed2c07e4c91c5eb" exitCode=6
Jul 18 01:37:52.527238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:52.527210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerDied","Data":"e852493ad57a94a98d74008eb6fc0caf331d1d6f2bfc979cfed2c07e4c91c5eb"}
Jul 18 01:37:52.527751 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:52.527316    2567 scope.go:117] "RemoveContainer" containerID="fd9904d8b48eb7de4b5604571bb6f138219257e85eed6f7e8b96d326071d7f15"
Jul 18 01:37:52.527751 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:52.527570    2567 scope.go:117] "RemoveContainer" containerID="e852493ad57a94a98d74008eb6fc0caf331d1d6f2bfc979cfed2c07e4c91c5eb"
Jul 18 01:37:52.527878 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:52.527827    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd_odh-ai-gateway-infra(168fcfa1-6846-4be8-801b-b5c583f34c01)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01"
Jul 18 01:37:53.535463 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:53.535436    2567 scope.go:117] "RemoveContainer" containerID="e852493ad57a94a98d74008eb6fc0caf331d1d6f2bfc979cfed2c07e4c91c5eb"
Jul 18 01:37:53.535875 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:37:53.535655    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd_odh-ai-gateway-infra(168fcfa1-6846-4be8-801b-b5c583f34c01)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01"
Jul 18 01:37:54.274149 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:54.274113    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:37:57.508320 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:57.508285    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:37:58.929951 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:37:58.929915    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:04.708016 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:04.707972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:04.782887 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:04.782851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:05.913895 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:05.913865    2567 scope.go:117] "RemoveContainer" containerID="e852493ad57a94a98d74008eb6fc0caf331d1d6f2bfc979cfed2c07e4c91c5eb"
Jul 18 01:38:05.914353 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:05.913973    2567 scope.go:117] "RemoveContainer" containerID="5b4c44c621aff3a19bf78f6d2bfea650e9a9a6c87c6451e2da685f85747e278a"
Jul 18 01:38:06.613812 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:06.613764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" event={"ID":"a4415239-3f29-4c90-9c76-a2ab4a22274d","Type":"ContainerStarted","Data":"cd144aec289c1f0b1eb7acf916dc1dd8265b4e7f969f0c61662771a39d08a949"}
Jul 18 01:38:06.615554 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:06.615529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerStarted","Data":"6e4b9f1ca862f394c3baec60770d9d6f8e7d14e112063d28ef257d838c01788e"}
Jul 18 01:38:06.628535 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:06.628475    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" podStartSLOduration=20.628458909 podStartE2EDuration="20.628458909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:37:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:38:06.627422561 +0000 UTC m=+3466.154927522" watchObservedRunningTime="2026-07-18 01:38:06.628458909 +0000 UTC m=+3466.155963869"
Jul 18 01:38:07.622698 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.622668    2567 generic.go:356] "Generic (PLEG): container finished" podID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerID="cd144aec289c1f0b1eb7acf916dc1dd8265b4e7f969f0c61662771a39d08a949" exitCode=6
Jul 18 01:38:07.623150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.622743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" event={"ID":"a4415239-3f29-4c90-9c76-a2ab4a22274d","Type":"ContainerDied","Data":"cd144aec289c1f0b1eb7acf916dc1dd8265b4e7f969f0c61662771a39d08a949"}
Jul 18 01:38:07.623150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.622802    2567 scope.go:117] "RemoveContainer" containerID="5b4c44c621aff3a19bf78f6d2bfea650e9a9a6c87c6451e2da685f85747e278a"
Jul 18 01:38:07.623150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.623065    2567 scope.go:117] "RemoveContainer" containerID="cd144aec289c1f0b1eb7acf916dc1dd8265b4e7f969f0c61662771a39d08a949"
Jul 18 01:38:07.623387 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:07.623364    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k_odh-ai-gateway-infra(a4415239-3f29-4c90-9c76-a2ab4a22274d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d"
Jul 18 01:38:07.624773 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.624754    2567 generic.go:356] "Generic (PLEG): container finished" podID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerID="6e4b9f1ca862f394c3baec60770d9d6f8e7d14e112063d28ef257d838c01788e" exitCode=6
Jul 18 01:38:07.624865 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.624781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerDied","Data":"6e4b9f1ca862f394c3baec60770d9d6f8e7d14e112063d28ef257d838c01788e"}
Jul 18 01:38:07.625022 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.625008    2567 scope.go:117] "RemoveContainer" containerID="6e4b9f1ca862f394c3baec60770d9d6f8e7d14e112063d28ef257d838c01788e"
Jul 18 01:38:07.625258 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:07.625236    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd_odh-ai-gateway-infra(168fcfa1-6846-4be8-801b-b5c583f34c01)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01"
Jul 18 01:38:07.637142 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.637120    2567 scope.go:117] "RemoveContainer" containerID="e852493ad57a94a98d74008eb6fc0caf331d1d6f2bfc979cfed2c07e4c91c5eb"
Jul 18 01:38:07.638325 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.638298    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"]
Jul 18 01:38:07.652912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:07.652887    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"]
Jul 18 01:38:08.799491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:08.799464    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:38:08.801461 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:08.801443    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:38:08.949878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:08.949788    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168fcfa1-6846-4be8-801b-b5c583f34c01-maas-api-service-ca\") pod \"168fcfa1-6846-4be8-801b-b5c583f34c01\" (UID: \"168fcfa1-6846-4be8-801b-b5c583f34c01\") "
Jul 18 01:38:08.949878 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:08.949875    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4415239-3f29-4c90-9c76-a2ab4a22274d-maas-api-service-ca\") pod \"a4415239-3f29-4c90-9c76-a2ab4a22274d\" (UID: \"a4415239-3f29-4c90-9c76-a2ab4a22274d\") "
Jul 18 01:38:08.950230 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:08.950208    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168fcfa1-6846-4be8-801b-b5c583f34c01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "168fcfa1-6846-4be8-801b-b5c583f34c01" (UID: "168fcfa1-6846-4be8-801b-b5c583f34c01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:08.950336 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:08.950238    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4415239-3f29-4c90-9c76-a2ab4a22274d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4415239-3f29-4c90-9c76-a2ab4a22274d" (UID: "a4415239-3f29-4c90-9c76-a2ab4a22274d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:09.050953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.050908    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4415239-3f29-4c90-9c76-a2ab4a22274d-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:09.050953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.050945    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168fcfa1-6846-4be8-801b-b5c583f34c01-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:09.625285 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.625229    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"]
Jul 18 01:38:09.625605 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.625574    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy" containerID="cri-o://362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc" gracePeriod=30
Jul 18 01:38:09.641089 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.641056    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k" event={"ID":"a4415239-3f29-4c90-9c76-a2ab4a22274d","Type":"ContainerDied","Data":"222c65bd119fdecca3863cdfb20c65bb3df09065c0e69ca1d1bbbb4b182cc724"}
Jul 18 01:38:09.641369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.641066    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"
Jul 18 01:38:09.641369 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.641131    2567 scope.go:117] "RemoveContainer" containerID="cd144aec289c1f0b1eb7acf916dc1dd8265b4e7f969f0c61662771a39d08a949"
Jul 18 01:38:09.642659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.642636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd" event={"ID":"168fcfa1-6846-4be8-801b-b5c583f34c01","Type":"ContainerDied","Data":"acbf02d4bc490bd4ed06012a2234d6fa106f9335e68058c6e4471fadaa097bbf"}
Jul 18 01:38:09.642745 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.642705    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"
Jul 18 01:38:09.648602 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.648574    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:09.653662 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.653635    2567 scope.go:117] "RemoveContainer" containerID="6e4b9f1ca862f394c3baec60770d9d6f8e7d14e112063d28ef257d838c01788e"
Jul 18 01:38:09.669737 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.669705    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"]
Jul 18 01:38:09.672807 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.672778    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-gqj9k"]
Jul 18 01:38:09.683502 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.683476    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"]
Jul 18 01:38:09.687020 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.686996    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-h7crd"]
Jul 18 01:38:09.862349 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.862312    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:09.985015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.984929    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"]
Jul 18 01:38:09.985503 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985486    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:09.985578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985507    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:09.985578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985521    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.985578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985530    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.985578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985557    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.985578 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985567    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985588    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985597    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985687    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985696    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985703    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985711    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.985744 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.985717    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:09.990234 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.990217    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:09.992607 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.992582    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:38:09.992758 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.992607    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:38:09.995972 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:09.995946    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"]
Jul 18 01:38:10.164402 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.164367    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab10ae1e-9e2b-4120-ad8a-84ba39d5480a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6\" (UID: \"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:10.265324 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.265218    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab10ae1e-9e2b-4120-ad8a-84ba39d5480a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6\" (UID: \"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:10.265874 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.265855    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab10ae1e-9e2b-4120-ad8a-84ba39d5480a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6\" (UID: \"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:10.304114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.304081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:10.450153 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.450123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"]
Jul 18 01:38:10.452254 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:38:10.452222    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab10ae1e_9e2b_4120_ad8a_84ba39d5480a.slice/crio-b8a976cb46e224fa4120e6ee0c8b3f5f2104e85a131b091fddb88cc09385a191 WatchSource:0}: Error finding container b8a976cb46e224fa4120e6ee0c8b3f5f2104e85a131b091fddb88cc09385a191: Status 404 returned error can't find the container with id b8a976cb46e224fa4120e6ee0c8b3f5f2104e85a131b091fddb88cc09385a191
Jul 18 01:38:10.508700 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.508668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:10.610712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.610682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:10.652531 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.652454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6" event={"ID":"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a","Type":"ContainerStarted","Data":"b8a976cb46e224fa4120e6ee0c8b3f5f2104e85a131b091fddb88cc09385a191"}
Jul 18 01:38:10.920114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.920036    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" path="/var/lib/kubelet/pods/168fcfa1-6846-4be8-801b-b5c583f34c01/volumes"
Jul 18 01:38:10.920491 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:10.920397    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" path="/var/lib/kubelet/pods/a4415239-3f29-4c90-9c76-a2ab4a22274d/volumes"
Jul 18 01:38:11.661688 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:11.661652    2567 generic.go:356] "Generic (PLEG): container finished" podID="ab10ae1e-9e2b-4120-ad8a-84ba39d5480a" containerID="ad163eb713d0ec261a0fae904fd0a0623d3d2a1a128e9f6e59ea8461b62bee87" exitCode=0
Jul 18 01:38:11.661877 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:11.661742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6" event={"ID":"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a","Type":"ContainerDied","Data":"ad163eb713d0ec261a0fae904fd0a0623d3d2a1a128e9f6e59ea8461b62bee87"}
Jul 18 01:38:12.812113 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:12.812087    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:12.992696 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:12.992596    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab10ae1e-9e2b-4120-ad8a-84ba39d5480a-maas-api-service-ca\") pod \"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a\" (UID: \"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a\") "
Jul 18 01:38:12.992984 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:12.992961    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab10ae1e-9e2b-4120-ad8a-84ba39d5480a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab10ae1e-9e2b-4120-ad8a-84ba39d5480a" (UID: "ab10ae1e-9e2b-4120-ad8a-84ba39d5480a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:13.093699 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:13.093659    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab10ae1e-9e2b-4120-ad8a-84ba39d5480a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:13.681858 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:13.681828    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6"
Jul 18 01:38:13.682082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:13.681874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6" event={"ID":"ab10ae1e-9e2b-4120-ad8a-84ba39d5480a","Type":"ContainerDied","Data":"b8a976cb46e224fa4120e6ee0c8b3f5f2104e85a131b091fddb88cc09385a191"}
Jul 18 01:38:13.682082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:13.681945    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8a976cb46e224fa4120e6ee0c8b3f5f2104e85a131b091fddb88cc09385a191"
Jul 18 01:38:14.900407 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:14.900334    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:38:15.009776 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009736    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-envoy\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009797    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istiod-ca-cert\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009832    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-data\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009853    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-socket\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009873    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxklq\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-kube-api-access-qxklq\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009912    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-certs\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009960    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-token\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.009981 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.009982    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-credential-socket\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.010448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010033    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-podinfo\") pod \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\" (UID: \"d2c31afe-b1b8-4c96-9598-0e44bc09a888\") "
Jul 18 01:38:15.010448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010194    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:15.010448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010250    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:15.010448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010399    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.010448 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010420    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.010706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010585    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:15.010706 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010618    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:15.010958 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.010935    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:15.012248 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.012223    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:15.012373 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.012351    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:38:15.012798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.012766    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:38:15.012798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.012766    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-kube-api-access-qxklq" (OuterVolumeSpecName: "kube-api-access-qxklq") pod "d2c31afe-b1b8-4c96-9598-0e44bc09a888" (UID: "d2c31afe-b1b8-4c96-9598-0e44bc09a888"). InnerVolumeSpecName "kube-api-access-qxklq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:38:15.111879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111839    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.111879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111871    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.111879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111881    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxklq\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-kube-api-access-qxklq\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.111879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111891    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.112181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111899    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.112181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111907    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c31afe-b1b8-4c96-9598-0e44bc09a888-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.112181 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.111914    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c31afe-b1b8-4c96-9598-0e44bc09a888-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:15.695652 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.695617    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerID="362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc" exitCode=0
Jul 18 01:38:15.695813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.695673    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"
Jul 18 01:38:15.695813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.695684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" event={"ID":"d2c31afe-b1b8-4c96-9598-0e44bc09a888","Type":"ContainerDied","Data":"362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc"}
Jul 18 01:38:15.695813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.695728    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44" event={"ID":"d2c31afe-b1b8-4c96-9598-0e44bc09a888","Type":"ContainerDied","Data":"df122b91bddcb6d26454c4fed7ac449b6ccd553515d8db2f7d74c54af21000f4"}
Jul 18 01:38:15.695813 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.695749    2567 scope.go:117] "RemoveContainer" containerID="362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc"
Jul 18 01:38:15.705965 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.705948    2567 scope.go:117] "RemoveContainer" containerID="1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"
Jul 18 01:38:15.715114 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.715096    2567 scope.go:117] "RemoveContainer" containerID="362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc"
Jul 18 01:38:15.715468 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:15.715442    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc\": container with ID starting with 362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc not found: ID does not exist" containerID="362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc"
Jul 18 01:38:15.715594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.715474    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc"} err="failed to get container status \"362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc\": rpc error: code = NotFound desc = could not find container \"362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc\": container with ID starting with 362ee73ec9aa311493c4c90d94bc5ef9096d9dfa4ae22f9acdea61a7dc5f5bfc not found: ID does not exist"
Jul 18 01:38:15.715594 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.715499    2567 scope.go:117] "RemoveContainer" containerID="1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"
Jul 18 01:38:15.715806 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:15.715788    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f\": container with ID starting with 1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f not found: ID does not exist" containerID="1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"
Jul 18 01:38:15.715883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.715813    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f"} err="failed to get container status \"1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f\": rpc error: code = NotFound desc = could not find container \"1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f\": container with ID starting with 1bb0c2ccc4b4b8f3820e041dee69128fde097621b940dc2baa7784e35dbb615f not found: ID does not exist"
Jul 18 01:38:15.717990 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.717968    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"]
Jul 18 01:38:15.721605 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:15.721582    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-6b3233-openshift-default-7946476559-h2d44"]
Jul 18 01:38:16.920284 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:16.920238    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" path="/var/lib/kubelet/pods/d2c31afe-b1b8-4c96-9598-0e44bc09a888/volumes"
Jul 18 01:38:20.425226 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:20.425193    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:23.512921 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:23.512887    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52mmm"]
Jul 18 01:38:25.070043 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:25.070007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:31.159376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.159343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:31.227875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.227821    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:31.755511 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755479    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"]
Jul 18 01:38:31.755900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755887    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy"
Jul 18 01:38:31.755950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755902    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy"
Jul 18 01:38:31.755950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755911    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:31.755950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755920    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:31.755950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755941    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy"
Jul 18 01:38:31.755950 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755948    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755957    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab10ae1e-9e2b-4120-ad8a-84ba39d5480a" containerName="revoke-keys"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755963    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab10ae1e-9e2b-4120-ad8a-84ba39d5480a" containerName="revoke-keys"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755976    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.755981    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="168fcfa1-6846-4be8-801b-b5c583f34c01" containerName="revoke-keys"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.756052    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.756062    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4415239-3f29-4c90-9c76-a2ab4a22274d" containerName="revoke-keys"
Jul 18 01:38:31.756186 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.756068    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab10ae1e-9e2b-4120-ad8a-84ba39d5480a" containerName="revoke-keys"
Jul 18 01:38:31.760883 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.760862    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:31.763835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.763793    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:38:31.763835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.763814    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:38:31.766240 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.766211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"]
Jul 18 01:38:31.862070 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.862028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8\" (UID: \"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:31.962930 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.962894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8\" (UID: \"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:31.963555 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:31.963534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8\" (UID: \"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:32.072872 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:32.072833    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:32.211678 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:32.211654    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"]
Jul 18 01:38:32.213705 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:38:32.213669    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d83beb_aaf5_4c1e_b0f2_f5ee1b75fb8a.slice/crio-abb9597e1f6a332af7dfe83fb5259fed1a3362fa168427965cd9d3754cb57480 WatchSource:0}: Error finding container abb9597e1f6a332af7dfe83fb5259fed1a3362fa168427965cd9d3754cb57480: Status 404 returned error can't find the container with id abb9597e1f6a332af7dfe83fb5259fed1a3362fa168427965cd9d3754cb57480
Jul 18 01:38:32.802733 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:32.802698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" event={"ID":"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a","Type":"ContainerStarted","Data":"bbbbcfea77622312ce3d9d3f68e22a314c00a838d84a5a91b4a0c842562192bf"}
Jul 18 01:38:32.802914 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:32.802754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" event={"ID":"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a","Type":"ContainerStarted","Data":"abb9597e1f6a332af7dfe83fb5259fed1a3362fa168427965cd9d3754cb57480"}
Jul 18 01:38:32.820210 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:32.820156    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" podStartSLOduration=1.8201384539999999 podStartE2EDuration="1.820138454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:38:32.817765462 +0000 UTC m=+3492.345276901" watchObservedRunningTime="2026-07-18 01:38:32.820138454 +0000 UTC m=+3492.347643414"
Jul 18 01:38:33.810217 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:33.810185    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerID="bbbbcfea77622312ce3d9d3f68e22a314c00a838d84a5a91b4a0c842562192bf" exitCode=6
Jul 18 01:38:33.810644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:33.810242    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" event={"ID":"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a","Type":"ContainerDied","Data":"bbbbcfea77622312ce3d9d3f68e22a314c00a838d84a5a91b4a0c842562192bf"}
Jul 18 01:38:33.810644 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:33.810493    2567 scope.go:117] "RemoveContainer" containerID="bbbbcfea77622312ce3d9d3f68e22a314c00a838d84a5a91b4a0c842562192bf"
Jul 18 01:38:34.205528 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.205441    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"]
Jul 18 01:38:34.205902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.205888    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2c31afe-b1b8-4c96-9598-0e44bc09a888" containerName="istio-proxy"
Jul 18 01:38:34.208913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.208894    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:34.216850 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.216309    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"]
Jul 18 01:38:34.287450 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.287410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7257d638-00fe-433b-a809-e739387a2673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4lmws\" (UID: \"7257d638-00fe-433b-a809-e739387a2673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:34.388174 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.388135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7257d638-00fe-433b-a809-e739387a2673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4lmws\" (UID: \"7257d638-00fe-433b-a809-e739387a2673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:34.388795 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.388775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7257d638-00fe-433b-a809-e739387a2673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4lmws\" (UID: \"7257d638-00fe-433b-a809-e739387a2673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:34.521363 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.521256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:34.662229 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.662205    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"]
Jul 18 01:38:34.664478 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:38:34.664445    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7257d638_00fe_433b_a809_e739387a2673.slice/crio-484f4d7873c10445282e0af80353347b96f75ac9e959c36725cb9f547eb81a5b WatchSource:0}: Error finding container 484f4d7873c10445282e0af80353347b96f75ac9e959c36725cb9f547eb81a5b: Status 404 returned error can't find the container with id 484f4d7873c10445282e0af80353347b96f75ac9e959c36725cb9f547eb81a5b
Jul 18 01:38:34.817548 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.817505    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" event={"ID":"7257d638-00fe-433b-a809-e739387a2673","Type":"ContainerStarted","Data":"484f4d7873c10445282e0af80353347b96f75ac9e959c36725cb9f547eb81a5b"}
Jul 18 01:38:34.819242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.819218    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerID="d97b12cf9bfdffbd720721feda04e084ff244a647aeeb48b0ba79a366882a2de" exitCode=6
Jul 18 01:38:34.819372 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.819249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" event={"ID":"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a","Type":"ContainerDied","Data":"d97b12cf9bfdffbd720721feda04e084ff244a647aeeb48b0ba79a366882a2de"}
Jul 18 01:38:34.819372 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.819303    2567 scope.go:117] "RemoveContainer" containerID="bbbbcfea77622312ce3d9d3f68e22a314c00a838d84a5a91b4a0c842562192bf"
Jul 18 01:38:34.819559 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:34.819543    2567 scope.go:117] "RemoveContainer" containerID="d97b12cf9bfdffbd720721feda04e084ff244a647aeeb48b0ba79a366882a2de"
Jul 18 01:38:34.819772 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:34.819754    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8_odh-ai-gateway-infra(b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a"
Jul 18 01:38:35.726384 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.726296    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"]
Jul 18 01:38:35.726563 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.726505    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" containerID="cri-o://ceca33f8824787962d0551c1d26311b99b34591fc34269b1b1cb053143c01f59" gracePeriod=30
Jul 18 01:38:35.742238 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.742198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:35.828049 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.828002    2567 generic.go:356] "Generic (PLEG): container finished" podID="7257d638-00fe-433b-a809-e739387a2673" containerID="36b88c754a119165ef2a7f2a0e47e5bc0be136ba2314f45a04d021c70de63ade" exitCode=6
Jul 18 01:38:35.828526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.828119    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" event={"ID":"7257d638-00fe-433b-a809-e739387a2673","Type":"ContainerDied","Data":"36b88c754a119165ef2a7f2a0e47e5bc0be136ba2314f45a04d021c70de63ade"}
Jul 18 01:38:35.828526 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.828420    2567 scope.go:117] "RemoveContainer" containerID="36b88c754a119165ef2a7f2a0e47e5bc0be136ba2314f45a04d021c70de63ade"
Jul 18 01:38:35.831241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.831217    2567 scope.go:117] "RemoveContainer" containerID="d97b12cf9bfdffbd720721feda04e084ff244a647aeeb48b0ba79a366882a2de"
Jul 18 01:38:35.831520 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:35.831491    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8_odh-ai-gateway-infra(b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a"
Jul 18 01:38:35.977034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:35.976941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:36.838481 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:36.838447    2567 generic.go:356] "Generic (PLEG): container finished" podID="7257d638-00fe-433b-a809-e739387a2673" containerID="ee92c15247ea912cae316efb1d8b0a572fbe3c790606dc6e0f977ded987d85eb" exitCode=6
Jul 18 01:38:36.838919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:36.838511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" event={"ID":"7257d638-00fe-433b-a809-e739387a2673","Type":"ContainerDied","Data":"ee92c15247ea912cae316efb1d8b0a572fbe3c790606dc6e0f977ded987d85eb"}
Jul 18 01:38:36.838919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:36.838599    2567 scope.go:117] "RemoveContainer" containerID="36b88c754a119165ef2a7f2a0e47e5bc0be136ba2314f45a04d021c70de63ade"
Jul 18 01:38:36.838919 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:36.838779    2567 scope.go:117] "RemoveContainer" containerID="ee92c15247ea912cae316efb1d8b0a572fbe3c790606dc6e0f977ded987d85eb"
Jul 18 01:38:36.839058 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:36.839008    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-4lmws_odh-ai-gateway-infra(7257d638-00fe-433b-a809-e739387a2673)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" podUID="7257d638-00fe-433b-a809-e739387a2673"
Jul 18 01:38:37.845870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:37.845842    2567 scope.go:117] "RemoveContainer" containerID="ee92c15247ea912cae316efb1d8b0a572fbe3c790606dc6e0f977ded987d85eb"
Jul 18 01:38:37.846395 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:37.846034    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-4lmws_odh-ai-gateway-infra(7257d638-00fe-433b-a809-e739387a2673)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" podUID="7257d638-00fe-433b-a809-e739387a2673"
Jul 18 01:38:39.038742 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:39.038704    2567 patch_prober.go:28] interesting pod/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 18 01:38:39.039193 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:39.038769    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 01:38:40.868120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:40.868082    2567 generic.go:356] "Generic (PLEG): container finished" podID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerID="ceca33f8824787962d0551c1d26311b99b34591fc34269b1b1cb053143c01f59" exitCode=0
Jul 18 01:38:40.868567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:40.868165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" event={"ID":"f690faca-2ef9-406b-a34d-96e24d9dd24d","Type":"ContainerDied","Data":"ceca33f8824787962d0551c1d26311b99b34591fc34269b1b1cb053143c01f59"}
Jul 18 01:38:40.868567 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:40.868249    2567 scope.go:117] "RemoveContainer" containerID="938996b8e2217a38ff44e72ec5e7c7e9bf932c5574d88cef7009eaf624efe5c2"
Jul 18 01:38:40.995650 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:40.995585    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:38:41.048757 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.048713    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f690faca-2ef9-406b-a34d-96e24d9dd24d-istiod-ca-cert\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.048968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.048769    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpbmb\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-kube-api-access-hpbmb\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.048968 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.048807    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-data\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.048983    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-socket\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049067    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-envoy\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049108 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049092    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-podinfo\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049145    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-credential-socket\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049170    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-certs\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049208    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f690faca-2ef9-406b-a34d-96e24d9dd24d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:41.049530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:41.049530 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049344    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-token\") pod \"f690faca-2ef9-406b-a34d-96e24d9dd24d\" (UID: \"f690faca-2ef9-406b-a34d-96e24d9dd24d\") "
Jul 18 01:38:41.049676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049653    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:41.049832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049754    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-credential-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.049832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049788    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f690faca-2ef9-406b-a34d-96e24d9dd24d-istiod-ca-cert\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.049832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049802    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-socket\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.049832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049797    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:41.049832 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.049815    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:41.051560 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.051526    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-kube-api-access-hpbmb" (OuterVolumeSpecName: "kube-api-access-hpbmb") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "kube-api-access-hpbmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:38:41.051676 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.051598    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:38:41.052072 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.052049    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 18 01:38:41.052183 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.052078    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f690faca-2ef9-406b-a34d-96e24d9dd24d" (UID: "f690faca-2ef9-406b-a34d-96e24d9dd24d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:38:41.150309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.150205    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-envoy\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.150309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.150237    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-podinfo\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.150309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.150247    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-workload-certs\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.150309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.150255    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-token\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.150309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.150263    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpbmb\" (UniqueName: \"kubernetes.io/projected/f690faca-2ef9-406b-a34d-96e24d9dd24d-kube-api-access-hpbmb\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.150309 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.150304    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f690faca-2ef9-406b-a34d-96e24d9dd24d-istio-data\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:41.789145 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.789115    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z8dkt_0972432b-1e47-4eea-9ba5-5d0ab0d94b55/manager/0.log"
Jul 18 01:38:41.881519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.881492    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"
Jul 18 01:38:41.881519 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.881511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p" event={"ID":"f690faca-2ef9-406b-a34d-96e24d9dd24d","Type":"ContainerDied","Data":"5b2064506cd71ab566f9b043817800fe79200e11d73d3d7e8cb7a6d1d3caa4ed"}
Jul 18 01:38:41.882031 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.881592    2567 scope.go:117] "RemoveContainer" containerID="ceca33f8824787962d0551c1d26311b99b34591fc34269b1b1cb053143c01f59"
Jul 18 01:38:41.903788 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.903760    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"]
Jul 18 01:38:41.907485 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:41.907460    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b17102-openshift-default-98bb6b65f-ch84p"]
Jul 18 01:38:42.153354 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:42.153330    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lkhmr_12dc7c62-a256-4a17-bee1-ed09820e3a01/server/0.log"
Jul 18 01:38:42.274093 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:42.274063    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gpxfc_66288b5c-9cce-42a9-a9ec-7132892da61e/manager/0.log"
Jul 18 01:38:42.506645 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:42.506565    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd8c7c4c4-9n688_3611f732-9623-40c4-bcbe-7bdbd511d053/copy-manifests/0.log"
Jul 18 01:38:42.518388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:42.518358    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd8c7c4c4-9n688_3611f732-9623-40c4-bcbe-7bdbd511d053/manager/0.log"
Jul 18 01:38:42.919141 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:42.919108    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" path="/var/lib/kubelet/pods/f690faca-2ef9-406b-a34d-96e24d9dd24d/volumes"
Jul 18 01:38:43.115231 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.115204    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738940-wdx9t_ffb0bb4a-d789-4fb4-babc-8f38ab93fa55/cleanup/0.log"
Jul 18 01:38:43.221636 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.221562    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738955-jxgcf_02928864-fdb6-41ae-af56-6ee7cb81dc39/cleanup/0.log"
Jul 18 01:38:43.334732 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.334710    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738970-mdwnb_1f6c655b-68a4-43b9-95e5-0579f579e9f6/cleanup/0.log"
Jul 18 01:38:43.445112 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.445080    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-354f15cb-4lmws_7257d638-00fe-433b-a809-e739387a2673/revoke-keys/1.log"
Jul 18 01:38:43.558306 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.558283    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-feded4-gxdkz_9de796c2-f9ae-426a-a7eb-6dae98935ec8/revoke-keys/0.log"
Jul 18 01:38:43.665929 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.665904    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-8775cd-96bnr_645ebe55-aec8-4145-bac5-a4c684761e7a/revoke-keys/0.log"
Jul 18 01:38:43.772754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.772715    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-1e72d0-zkxbn_bdf5d8f5-802c-4773-8327-65085b7f8ea6/revoke-keys/0.log"
Jul 18 01:38:43.876475 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.876378    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-8e3c6c-nrgng_273feba5-634a-4b53-8d3e-bf57e12f2800/revoke-keys/0.log"
Jul 18 01:38:43.987595 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:43.987560    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8_b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a/revoke-keys/1.log"
Jul 18 01:38:44.094536 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:44.094506    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-8d0c59be-cb6xt_a7589b4a-d1ad-474f-b053-f350da25f0c9/revoke-keys/0.log"
Jul 18 01:38:44.207082 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:44.206992    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-f92ba3bc-svbf9_d717c6f4-08b7-4fbf-a7a4-26672223aa2a/revoke-keys/0.log"
Jul 18 01:38:44.316685 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:44.316657    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-6b3233-hqpx5_2b9783e0-2647-4e89-af7e-04f2215e48d8/revoke-keys/0.log"
Jul 18 01:38:44.424922 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:44.424892    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-b17102-2wcc6_ab10ae1e-9e2b-4120-ad8a-84ba39d5480a/revoke-keys/0.log"
Jul 18 01:38:46.099700 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.099673    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4qgrv_3941225a-1acf-4693-8031-e26fd4904303/manager/0.log"
Jul 18 01:38:46.211079 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.211053    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fslf6_c54113c7-77bf-4685-b1bb-06b267b7c8cb/kuadrant-console-plugin/0.log"
Jul 18 01:38:46.328138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.328106    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pm4kd_cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d/registry-server/0.log"
Jul 18 01:38:46.447571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.447474    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lt4s2_8aa691d8-5b45-4e33-9305-c0d67372ea18/manager/0.log"
Jul 18 01:38:46.555879 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.555845    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-52mmm_09578043-ec7d-4407-9345-b15116d06fcf/limitador/0.log"
Jul 18 01:38:46.668557 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.668526    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xpj7w_cf8d77a7-8d93-4cc4-91bd-686536ed9158/manager/0.log"
Jul 18 01:38:46.818062 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.818023    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"]
Jul 18 01:38:46.818468 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.818454    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy"
Jul 18 01:38:46.818539 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.818470    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy"
Jul 18 01:38:46.818581 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.818545    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy"
Jul 18 01:38:46.823076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.823051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:38:46.828312 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.828259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"]
Jul 18 01:38:46.905434 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:46.905404    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq\" (UID: \"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:38:47.006161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.006118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq\" (UID: \"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:38:47.006819 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.006795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq\" (UID: \"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:38:47.031609 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.031577    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qgn9p_76d97c25-5c24-4f78-92a8-e1c87e2f0298/istio-proxy/0.log"
Jul 18 01:38:47.134931 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.134835    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:38:47.274576 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.274549    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"]
Jul 18 01:38:47.276125 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:38:47.276099    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab3043f_5b7b_44a0_a6b7_861d5f3361ea.slice/crio-8282383e77b56abd60267a92257a969e7204140cdf73361b98ad56e9997fd582 WatchSource:0}: Error finding container 8282383e77b56abd60267a92257a969e7204140cdf73361b98ad56e9997fd582: Status 404 returned error can't find the container with id 8282383e77b56abd60267a92257a969e7204140cdf73361b98ad56e9997fd582
Jul 18 01:38:47.374424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.374376    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7569c6bc9b-vmpgv_24c8026f-5014-4d63-95a0-5d46060e2ecc/kube-auth-proxy/0.log"
Jul 18 01:38:47.491794 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.491769    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/1.log"
Jul 18 01:38:47.831394 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.831364    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7df7df96f4-vqk8n_7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608/router/0.log"
Jul 18 01:38:47.913103 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.913078    2567 scope.go:117] "RemoveContainer" containerID="d97b12cf9bfdffbd720721feda04e084ff244a647aeeb48b0ba79a366882a2de"
Jul 18 01:38:47.916810 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.916788    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerID="a7f6d20afb77f2f402ca38429b75cdf8bf2fc1dc3b3a0854af6fdc77b2d1bce6" exitCode=6
Jul 18 01:38:47.916932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.916868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" event={"ID":"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea","Type":"ContainerDied","Data":"a7f6d20afb77f2f402ca38429b75cdf8bf2fc1dc3b3a0854af6fdc77b2d1bce6"}
Jul 18 01:38:47.916932 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.916918    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" event={"ID":"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea","Type":"ContainerStarted","Data":"8282383e77b56abd60267a92257a969e7204140cdf73361b98ad56e9997fd582"}
Jul 18 01:38:47.917047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:47.917024    2567 scope.go:117] "RemoveContainer" containerID="a7f6d20afb77f2f402ca38429b75cdf8bf2fc1dc3b3a0854af6fdc77b2d1bce6"
Jul 18 01:38:48.913731 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.913699    2567 scope.go:117] "RemoveContainer" containerID="ee92c15247ea912cae316efb1d8b0a572fbe3c790606dc6e0f977ded987d85eb"
Jul 18 01:38:48.926959 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.926928    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerID="025b4aba49a6a18d9f84598f86ab4a9fcf960aa3bfb0467066975ed6745b790e" exitCode=6
Jul 18 01:38:48.927150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.927013    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" event={"ID":"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a","Type":"ContainerDied","Data":"025b4aba49a6a18d9f84598f86ab4a9fcf960aa3bfb0467066975ed6745b790e"}
Jul 18 01:38:48.927150 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.927092    2567 scope.go:117] "RemoveContainer" containerID="d97b12cf9bfdffbd720721feda04e084ff244a647aeeb48b0ba79a366882a2de"
Jul 18 01:38:48.927587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.927454    2567 scope.go:117] "RemoveContainer" containerID="025b4aba49a6a18d9f84598f86ab4a9fcf960aa3bfb0467066975ed6745b790e"
Jul 18 01:38:48.927927 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:48.927886    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8_odh-ai-gateway-infra(b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a"
Jul 18 01:38:48.929347 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.929319    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerID="a3885aff44aade33ac22776f9dec1a1c401b489ef1c08f4af26485ceeb7990da" exitCode=6
Jul 18 01:38:48.929469 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.929400    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" event={"ID":"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea","Type":"ContainerDied","Data":"a3885aff44aade33ac22776f9dec1a1c401b489ef1c08f4af26485ceeb7990da"}
Jul 18 01:38:48.929654 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.929640    2567 scope.go:117] "RemoveContainer" containerID="a3885aff44aade33ac22776f9dec1a1c401b489ef1c08f4af26485ceeb7990da"
Jul 18 01:38:48.929845 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:48.929827    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq_odh-ai-gateway-infra(8ab3043f-5b7b-44a0-a6b7-861d5f3361ea)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea"
Jul 18 01:38:48.941738 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:48.941715    2567 scope.go:117] "RemoveContainer" containerID="a7f6d20afb77f2f402ca38429b75cdf8bf2fc1dc3b3a0854af6fdc77b2d1bce6"
Jul 18 01:38:49.578162 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.578120    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"]
Jul 18 01:38:49.578638 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.578618    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy"
Jul 18 01:38:49.578743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.578640    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy"
Jul 18 01:38:49.578801 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.578760    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f690faca-2ef9-406b-a34d-96e24d9dd24d" containerName="istio-proxy"
Jul 18 01:38:49.582006 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.581984    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:38:49.588058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.588023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"]
Jul 18 01:38:49.729797 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.729759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73f82275-fb2c-470a-b4e8-9ef682fe8dfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v\" (UID: \"73f82275-fb2c-470a-b4e8-9ef682fe8dfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:38:49.830570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.830480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73f82275-fb2c-470a-b4e8-9ef682fe8dfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v\" (UID: \"73f82275-fb2c-470a-b4e8-9ef682fe8dfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:38:49.831045 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.831025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73f82275-fb2c-470a-b4e8-9ef682fe8dfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v\" (UID: \"73f82275-fb2c-470a-b4e8-9ef682fe8dfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:38:49.894039 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.894005    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:38:49.939242 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.939212    2567 generic.go:356] "Generic (PLEG): container finished" podID="7257d638-00fe-433b-a809-e739387a2673" containerID="e079c2c1fcc29b281ff9cb40d96015d9252f605f50dbcd0d902767a434e8b768" exitCode=6
Jul 18 01:38:49.939730 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.939303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" event={"ID":"7257d638-00fe-433b-a809-e739387a2673","Type":"ContainerDied","Data":"e079c2c1fcc29b281ff9cb40d96015d9252f605f50dbcd0d902767a434e8b768"}
Jul 18 01:38:49.939730 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.939364    2567 scope.go:117] "RemoveContainer" containerID="ee92c15247ea912cae316efb1d8b0a572fbe3c790606dc6e0f977ded987d85eb"
Jul 18 01:38:49.939730 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.939723    2567 scope.go:117] "RemoveContainer" containerID="e079c2c1fcc29b281ff9cb40d96015d9252f605f50dbcd0d902767a434e8b768"
Jul 18 01:38:49.939983 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:49.939957    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-354f15cb-4lmws_odh-ai-gateway-infra(7257d638-00fe-433b-a809-e739387a2673)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" podUID="7257d638-00fe-433b-a809-e739387a2673"
Jul 18 01:38:49.942865 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.942849    2567 scope.go:117] "RemoveContainer" containerID="a3885aff44aade33ac22776f9dec1a1c401b489ef1c08f4af26485ceeb7990da"
Jul 18 01:38:49.943061 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:49.943042    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq_odh-ai-gateway-infra(8ab3043f-5b7b-44a0-a6b7-861d5f3361ea)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea"
Jul 18 01:38:49.959043 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:49.957876    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"]
Jul 18 01:38:50.043778 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.043734    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"]
Jul 18 01:38:50.047847 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:38:50.047820    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73f82275_fb2c_470a_b4e8_9ef682fe8dfd.slice/crio-91fff93279c561b5e9f4b7dd767579855e9cc74322b195d6b6f4666918a84b9d WatchSource:0}: Error finding container 91fff93279c561b5e9f4b7dd767579855e9cc74322b195d6b6f4666918a84b9d: Status 404 returned error can't find the container with id 91fff93279c561b5e9f4b7dd767579855e9cc74322b195d6b6f4666918a84b9d
Jul 18 01:38:50.102087 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.102062    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:50.234468 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.234430    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a-maas-api-service-ca\") pod \"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a\" (UID: \"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a\") "
Jul 18 01:38:50.234807 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.234785    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" (UID: "b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:50.335424 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.335390    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:50.950829 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.950799    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"
Jul 18 01:38:50.951310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.950832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8" event={"ID":"b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a","Type":"ContainerDied","Data":"abb9597e1f6a332af7dfe83fb5259fed1a3362fa168427965cd9d3754cb57480"}
Jul 18 01:38:50.951310 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.950889    2567 scope.go:117] "RemoveContainer" containerID="025b4aba49a6a18d9f84598f86ab4a9fcf960aa3bfb0467066975ed6745b790e"
Jul 18 01:38:50.952630 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.952604    2567 generic.go:356] "Generic (PLEG): container finished" podID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerID="73e6c323ef6aa26173d505a0e5abeaa49896a654f61136f3b6384c8279b01e18" exitCode=6
Jul 18 01:38:50.952767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.952681    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" event={"ID":"73f82275-fb2c-470a-b4e8-9ef682fe8dfd","Type":"ContainerDied","Data":"73e6c323ef6aa26173d505a0e5abeaa49896a654f61136f3b6384c8279b01e18"}
Jul 18 01:38:50.952767 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.952747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" event={"ID":"73f82275-fb2c-470a-b4e8-9ef682fe8dfd","Type":"ContainerStarted","Data":"91fff93279c561b5e9f4b7dd767579855e9cc74322b195d6b6f4666918a84b9d"}
Jul 18 01:38:50.952913 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.952891    2567 scope.go:117] "RemoveContainer" containerID="73e6c323ef6aa26173d505a0e5abeaa49896a654f61136f3b6384c8279b01e18"
Jul 18 01:38:50.968076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.968050    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"]
Jul 18 01:38:50.975220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.975190    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"]
Jul 18 01:38:50.981802 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:50.981776    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-pjrs8"]
Jul 18 01:38:51.108003 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.107977    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:51.243432 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.243336    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7257d638-00fe-433b-a809-e739387a2673-maas-api-service-ca\") pod \"7257d638-00fe-433b-a809-e739387a2673\" (UID: \"7257d638-00fe-433b-a809-e739387a2673\") "
Jul 18 01:38:51.243753 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.243724    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7257d638-00fe-433b-a809-e739387a2673-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7257d638-00fe-433b-a809-e739387a2673" (UID: "7257d638-00fe-433b-a809-e739387a2673"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:38:51.344281 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.344242    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7257d638-00fe-433b-a809-e739387a2673-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:38:51.960341 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.960227    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"
Jul 18 01:38:51.960341 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.960236    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws" event={"ID":"7257d638-00fe-433b-a809-e739387a2673","Type":"ContainerDied","Data":"484f4d7873c10445282e0af80353347b96f75ac9e959c36725cb9f547eb81a5b"}
Jul 18 01:38:51.960341 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.960328    2567 scope.go:117] "RemoveContainer" containerID="e079c2c1fcc29b281ff9cb40d96015d9252f605f50dbcd0d902767a434e8b768"
Jul 18 01:38:51.963473 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.963432    2567 generic.go:356] "Generic (PLEG): container finished" podID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerID="c884d09fe976d463e95ef7310686790facc35bf05cfa186f6f04296163c5c28b" exitCode=6
Jul 18 01:38:51.963634 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.963524    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" event={"ID":"73f82275-fb2c-470a-b4e8-9ef682fe8dfd","Type":"ContainerDied","Data":"c884d09fe976d463e95ef7310686790facc35bf05cfa186f6f04296163c5c28b"}
Jul 18 01:38:51.963835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.963816    2567 scope.go:117] "RemoveContainer" containerID="c884d09fe976d463e95ef7310686790facc35bf05cfa186f6f04296163c5c28b"
Jul 18 01:38:51.964096 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:51.964070    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v_odh-ai-gateway-infra(73f82275-fb2c-470a-b4e8-9ef682fe8dfd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd"
Jul 18 01:38:51.972404 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.972338    2567 scope.go:117] "RemoveContainer" containerID="73e6c323ef6aa26173d505a0e5abeaa49896a654f61136f3b6384c8279b01e18"
Jul 18 01:38:51.991570 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.991496    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"]
Jul 18 01:38:51.995130 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:51.995099    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4lmws"]
Jul 18 01:38:52.920388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:52.920357    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7257d638-00fe-433b-a809-e739387a2673" path="/var/lib/kubelet/pods/7257d638-00fe-433b-a809-e739387a2673/volumes"
Jul 18 01:38:52.920717 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:52.920704    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" path="/var/lib/kubelet/pods/b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a/volumes"
Jul 18 01:38:52.972057 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:52.972028    2567 scope.go:117] "RemoveContainer" containerID="c884d09fe976d463e95ef7310686790facc35bf05cfa186f6f04296163c5c28b"
Jul 18 01:38:52.972461 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:38:52.972242    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v_odh-ai-gateway-infra(73f82275-fb2c-470a-b4e8-9ef682fe8dfd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd"
Jul 18 01:38:54.445228 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445190    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dn4z4/must-gather-4pbkg"]
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445641    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445654    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445668    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445673    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445698    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445704    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445710    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445715    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445723    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.445723 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445729    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445795    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445804    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445812    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445819    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445825    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445833    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d83beb-aaf5-4c1e-b0f2-f5ee1b75fb8a" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445888    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.446081 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.445893    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7257d638-00fe-433b-a809-e739387a2673" containerName="revoke-keys"
Jul 18 01:38:54.450505 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.450485    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.452613 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.452585    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dn4z4\"/\"openshift-service-ca.crt\""
Jul 18 01:38:54.452759 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.452640    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dn4z4\"/\"kube-root-ca.crt\""
Jul 18 01:38:54.456748 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.456721    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4z4/must-gather-4pbkg"]
Jul 18 01:38:54.460587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.460561    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:54.522204 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.522175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log" limit=1024
Jul 18 01:38:54.574458 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.574420    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/83c59d4a-4d24-4a93-85b5-c42487f2227d-must-gather-output\") pod \"must-gather-4pbkg\" (UID: \"83c59d4a-4d24-4a93-85b5-c42487f2227d\") " pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.574647 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.574474    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6g4v\" (UniqueName: \"kubernetes.io/projected/83c59d4a-4d24-4a93-85b5-c42487f2227d-kube-api-access-l6g4v\") pod \"must-gather-4pbkg\" (UID: \"83c59d4a-4d24-4a93-85b5-c42487f2227d\") " pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.675084 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.675036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/83c59d4a-4d24-4a93-85b5-c42487f2227d-must-gather-output\") pod \"must-gather-4pbkg\" (UID: \"83c59d4a-4d24-4a93-85b5-c42487f2227d\") " pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.675305 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.675100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g4v\" (UniqueName: \"kubernetes.io/projected/83c59d4a-4d24-4a93-85b5-c42487f2227d-kube-api-access-l6g4v\") pod \"must-gather-4pbkg\" (UID: \"83c59d4a-4d24-4a93-85b5-c42487f2227d\") " pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.675380 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.675363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/83c59d4a-4d24-4a93-85b5-c42487f2227d-must-gather-output\") pod \"must-gather-4pbkg\" (UID: \"83c59d4a-4d24-4a93-85b5-c42487f2227d\") " pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.685058 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.685030    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6g4v\" (UniqueName: \"kubernetes.io/projected/83c59d4a-4d24-4a93-85b5-c42487f2227d-kube-api-access-l6g4v\") pod \"must-gather-4pbkg\" (UID: \"83c59d4a-4d24-4a93-85b5-c42487f2227d\") " pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.760835 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.760744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4z4/must-gather-4pbkg"
Jul 18 01:38:54.906711 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.906685    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4z4/must-gather-4pbkg"]
Jul 18 01:38:54.909252 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:38:54.909222    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83c59d4a_4d24_4a93_85b5_c42487f2227d.slice/crio-552a4a26859fb5a14763e8e37714fec0a732990f6f65cf329927d87e8d327875 WatchSource:0}: Error finding container 552a4a26859fb5a14763e8e37714fec0a732990f6f65cf329927d87e8d327875: Status 404 returned error can't find the container with id 552a4a26859fb5a14763e8e37714fec0a732990f6f65cf329927d87e8d327875
Jul 18 01:38:54.911497 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.911479    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:38:54.985684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:54.985649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4z4/must-gather-4pbkg" event={"ID":"83c59d4a-4d24-4a93-85b5-c42487f2227d","Type":"ContainerStarted","Data":"552a4a26859fb5a14763e8e37714fec0a732990f6f65cf329927d87e8d327875"}
Jul 18 01:38:55.997575 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:55.997467    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4z4/must-gather-4pbkg" event={"ID":"83c59d4a-4d24-4a93-85b5-c42487f2227d","Type":"ContainerStarted","Data":"668a249625a03bf24072da68373883965dc0c2b77d50f40237307f84a1be82cb"}
Jul 18 01:38:55.998609 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:55.998572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4z4/must-gather-4pbkg" event={"ID":"83c59d4a-4d24-4a93-85b5-c42487f2227d","Type":"ContainerStarted","Data":"6ab3800625d23f2f63187ccece67b096cf110ff15b32ea5680b5a0b68e594d2c"}
Jul 18 01:38:56.016915 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:56.016851    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dn4z4/must-gather-4pbkg" podStartSLOduration=1.286625858 podStartE2EDuration="2.016827706s" totalImagesPullingTime="730.201848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:38:56.015146939 +0000 UTC m=+3515.542651902" watchObservedRunningTime="2026-07-18 01:38:56.016827706 +0000 UTC m=+3515.544332668"
Jul 18 01:38:57.189428 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:57.189398    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4bl8d_d0098140-e04e-4d1c-b440-09c25f6825f9/global-pull-secret-syncer/0.log"
Jul 18 01:38:57.322912 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:57.322882    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8rx49_842c946c-ddda-421a-ac48-d47849301274/konnectivity-agent/0.log"
Jul 18 01:38:57.450233 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:38:57.450144    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-55.ec2.internal_f0f668c5d34062566082260ffcecd15d/haproxy/0.log"
Jul 18 01:39:00.923410 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:00.923376    2567 scope.go:117] "RemoveContainer" containerID="a3885aff44aade33ac22776f9dec1a1c401b489ef1c08f4af26485ceeb7990da"
Jul 18 01:39:02.055759 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.055711    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerID="872dc99e3fe571d12e6d321a6e4f289fba50767ce4dc2b54e2497d2cb2c73c6e" exitCode=6
Jul 18 01:39:02.056456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.055890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" event={"ID":"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea","Type":"ContainerDied","Data":"872dc99e3fe571d12e6d321a6e4f289fba50767ce4dc2b54e2497d2cb2c73c6e"}
Jul 18 01:39:02.056456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.055952    2567 scope.go:117] "RemoveContainer" containerID="a3885aff44aade33ac22776f9dec1a1c401b489ef1c08f4af26485ceeb7990da"
Jul 18 01:39:02.056456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.056391    2567 scope.go:117] "RemoveContainer" containerID="872dc99e3fe571d12e6d321a6e4f289fba50767ce4dc2b54e2497d2cb2c73c6e"
Jul 18 01:39:02.056691 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:39:02.056670    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq_odh-ai-gateway-infra(8ab3043f-5b7b-44a0-a6b7-861d5f3361ea)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea"
Jul 18 01:39:02.522408 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.522374    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4qgrv_3941225a-1acf-4693-8031-e26fd4904303/manager/0.log"
Jul 18 01:39:02.557364 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.557197    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fslf6_c54113c7-77bf-4685-b1bb-06b267b7c8cb/kuadrant-console-plugin/0.log"
Jul 18 01:39:02.619300 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.619244    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pm4kd_cf441afe-bfe3-4e0d-a8ef-2b41f7c02e4d/registry-server/0.log"
Jul 18 01:39:02.791220 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.791110    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lt4s2_8aa691d8-5b45-4e33-9305-c0d67372ea18/manager/0.log"
Jul 18 01:39:02.816600 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.816575    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-52mmm_09578043-ec7d-4407-9345-b15116d06fcf/limitador/0.log"
Jul 18 01:39:02.906743 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:02.906711    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xpj7w_cf8d77a7-8d93-4cc4-91bd-686536ed9158/manager/0.log"
Jul 18 01:39:03.085241 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:03.085198    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"]
Jul 18 01:39:03.341343 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:03.341172    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:39:03.397510 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:03.397473    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea-maas-api-service-ca\") pod \"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea\" (UID: \"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea\") "
Jul 18 01:39:03.397906 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:03.397866    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" (UID: "8ab3043f-5b7b-44a0-a6b7-861d5f3361ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:39:03.499640 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:03.499585    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:39:04.080155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:04.080112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq" event={"ID":"8ab3043f-5b7b-44a0-a6b7-861d5f3361ea","Type":"ContainerDied","Data":"8282383e77b56abd60267a92257a969e7204140cdf73361b98ad56e9997fd582"}
Jul 18 01:39:04.080392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:04.080190    2567 scope.go:117] "RemoveContainer" containerID="872dc99e3fe571d12e6d321a6e4f289fba50767ce4dc2b54e2497d2cb2c73c6e"
Jul 18 01:39:04.080392 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:04.080352    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"
Jul 18 01:39:04.118577 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:04.118516    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"]
Jul 18 01:39:04.123556 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:04.123527    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8942d9e2-hf5nq"]
Jul 18 01:39:04.924487 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:04.924451    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" path="/var/lib/kubelet/pods/8ab3043f-5b7b-44a0-a6b7-861d5f3361ea/volumes"
Jul 18 01:39:05.036563 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:05.036439    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-4g6jq_7c5c65bb-51cb-4ffa-badb-a0c852c4b366/cluster-monitoring-operator/0.log"
Jul 18 01:39:05.330623 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:05.330593    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nsf2c_4b412783-2ae1-418a-9f27-42076704b6e5/node-exporter/0.log"
Jul 18 01:39:05.351693 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:05.351663    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nsf2c_4b412783-2ae1-418a-9f27-42076704b6e5/kube-rbac-proxy/0.log"
Jul 18 01:39:05.375790 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:05.375758    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nsf2c_4b412783-2ae1-418a-9f27-42076704b6e5/init-textfile/0.log"
Jul 18 01:39:06.919249 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:06.919220    2567 scope.go:117] "RemoveContainer" containerID="c884d09fe976d463e95ef7310686790facc35bf05cfa186f6f04296163c5c28b"
Jul 18 01:39:07.037479 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.037450    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-sdkv4_99bcb8a3-01c9-454e-ad27-17bd14fde331/networking-console-plugin/0.log"
Jul 18 01:39:07.509076 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.509041    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"]
Jul 18 01:39:07.509924 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.509899    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510097 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510084    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510206 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510194    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510316 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510305    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510418 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510408    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510506 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510496    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510709 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510694    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510803 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510784    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.510890 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.510803    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab3043f-5b7b-44a0-a6b7-861d5f3361ea" containerName="revoke-keys"
Jul 18 01:39:07.516462 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.516437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.518746 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.518725    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dn4z4\"/\"default-dockercfg-glxnk\""
Jul 18 01:39:07.520617 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.520379    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"]
Jul 18 01:39:07.654024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.653920    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-sys\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.654024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.653975    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-lib-modules\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.654024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.654001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-proc\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.654024 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.654026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-podres\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.654471 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.654108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzr4p\" (UniqueName: \"kubernetes.io/projected/b96ef97c-f530-43dd-b29d-d944eac0707a-kube-api-access-lzr4p\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.703659 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.703634    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/2.log"
Jul 18 01:39:07.709902 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.709859    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rrr5c_4dab6681-22ef-4518-9bea-f894457cd197/console-operator/3.log"
Jul 18 01:39:07.755209 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-sys\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755527 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-lib-modules\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755362    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-sys\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755537    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-proc\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755563    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-podres\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755617    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-proc\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755663 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzr4p\" (UniqueName: \"kubernetes.io/projected/b96ef97c-f530-43dd-b29d-d944eac0707a-kube-api-access-lzr4p\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755892 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-lib-modules\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.755892 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.755739    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b96ef97c-f530-43dd-b29d-d944eac0707a-podres\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.763863 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.763832    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzr4p\" (UniqueName: \"kubernetes.io/projected/b96ef97c-f530-43dd-b29d-d944eac0707a-kube-api-access-lzr4p\") pod \"perf-node-gather-daemonset-skv45\" (UID: \"b96ef97c-f530-43dd-b29d-d944eac0707a\") " pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:07.832077 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:07.832031    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:08.054817 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.054394    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"]
Jul 18 01:39:08.120215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.119119    2567 generic.go:356] "Generic (PLEG): container finished" podID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerID="67b80b98d235902832f28785ae81769a0023112fdd3787579aec7aa9634c5a8f" exitCode=6
Jul 18 01:39:08.120215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.119216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" event={"ID":"73f82275-fb2c-470a-b4e8-9ef682fe8dfd","Type":"ContainerDied","Data":"67b80b98d235902832f28785ae81769a0023112fdd3787579aec7aa9634c5a8f"}
Jul 18 01:39:08.120215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.119294    2567 scope.go:117] "RemoveContainer" containerID="c884d09fe976d463e95ef7310686790facc35bf05cfa186f6f04296163c5c28b"
Jul 18 01:39:08.120215 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.119673    2567 scope.go:117] "RemoveContainer" containerID="67b80b98d235902832f28785ae81769a0023112fdd3787579aec7aa9634c5a8f"
Jul 18 01:39:08.120215 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:39:08.119911    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v_odh-ai-gateway-infra(73f82275-fb2c-470a-b4e8-9ef682fe8dfd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd"
Jul 18 01:39:08.123990 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.123926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45" event={"ID":"b96ef97c-f530-43dd-b29d-d944eac0707a","Type":"ContainerStarted","Data":"7d62ea60ebce3c4b0a925b4af48401770d15cbcd88938c5f23b36158ac69491f"}
Jul 18 01:39:08.913629 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:08.913594    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-sgzwt_1d015041-2e4e-4f50-9bd1-ea921680c85d/volume-data-source-validator/0.log"
Jul 18 01:39:09.135532 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.135488    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45" event={"ID":"b96ef97c-f530-43dd-b29d-d944eac0707a","Type":"ContainerStarted","Data":"26c569b3fabd4ceb75f067723ac1d882f053588a26613d9bdbba7811dbc1f8bb"}
Jul 18 01:39:09.136041 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.135872    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:09.145335 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.145306    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"]
Jul 18 01:39:09.155633 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.155571    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45" podStartSLOduration=2.15554961 podStartE2EDuration="2.15554961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:39:09.153314273 +0000 UTC m=+3528.680819235" watchObservedRunningTime="2026-07-18 01:39:09.15554961 +0000 UTC m=+3528.683054571"
Jul 18 01:39:09.335027 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.334791    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:39:09.472953 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.472867    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73f82275-fb2c-470a-b4e8-9ef682fe8dfd-maas-api-service-ca\") pod \"73f82275-fb2c-470a-b4e8-9ef682fe8dfd\" (UID: \"73f82275-fb2c-470a-b4e8-9ef682fe8dfd\") "
Jul 18 01:39:09.473257 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.473229    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73f82275-fb2c-470a-b4e8-9ef682fe8dfd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73f82275-fb2c-470a-b4e8-9ef682fe8dfd" (UID: "73f82275-fb2c-470a-b4e8-9ef682fe8dfd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:39:09.574247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:09.574203    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73f82275-fb2c-470a-b4e8-9ef682fe8dfd-maas-api-service-ca\") on node \"ip-10-0-135-55.ec2.internal\" DevicePath \"\""
Jul 18 01:39:10.000194 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.000165    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8p9z_666e0686-d4c4-4d89-8ff5-e7de233831be/dns/0.log"
Jul 18 01:39:10.020388 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.020354    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8p9z_666e0686-d4c4-4d89-8ff5-e7de233831be/kube-rbac-proxy/0.log"
Jul 18 01:39:10.137728 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.137675    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdrwg_a8aa0034-92b0-46ce-9347-71f922623496/dns-node-resolver/0.log"
Jul 18 01:39:10.143192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.143155    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"
Jul 18 01:39:10.143192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.143180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v" event={"ID":"73f82275-fb2c-470a-b4e8-9ef682fe8dfd","Type":"ContainerDied","Data":"91fff93279c561b5e9f4b7dd767579855e9cc74322b195d6b6f4666918a84b9d"}
Jul 18 01:39:10.143435 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.143244    2567 scope.go:117] "RemoveContainer" containerID="67b80b98d235902832f28785ae81769a0023112fdd3787579aec7aa9634c5a8f"
Jul 18 01:39:10.170573 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.170528    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"]
Jul 18 01:39:10.174900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.174870    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d5466ad4-6jg9v"]
Jul 18 01:39:10.753943 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.753911    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fwvdq_0ed34f12-7e48-43c6-b50d-bbe40913298b/node-ca/0.log"
Jul 18 01:39:10.923198 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:10.923163    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" path="/var/lib/kubelet/pods/73f82275-fb2c-470a-b4e8-9ef682fe8dfd/volumes"
Jul 18 01:39:11.737101 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:11.737073    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qgn9p_76d97c25-5c24-4f78-92a8-e1c87e2f0298/istio-proxy/0.log"
Jul 18 01:39:11.906486 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:11.906456    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7569c6bc9b-vmpgv_24c8026f-5014-4d63-95a0-5d46060e2ecc/kube-auth-proxy/0.log"
Jul 18 01:39:11.937131 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:11.937063    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/0.log"
Jul 18 01:39:11.974376 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:11.974340    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/1.log"
Jul 18 01:39:12.056161 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:12.056096    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7df7df96f4-vqk8n_7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608/router/0.log"
Jul 18 01:39:12.735547 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:12.735518    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6m5bf_6ed78294-e7f5-44af-966b-8ba2d4d6821f/serve-healthcheck-canary/0.log"
Jul 18 01:39:13.240825 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:13.240775    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-km4hb_c4238d52-036a-4948-88cd-3d2f814d369c/insights-operator/0.log"
Jul 18 01:39:13.242047 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:13.242020    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-km4hb_c4238d52-036a-4948-88cd-3d2f814d369c/insights-operator/1.log"
Jul 18 01:39:13.263209 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:13.263173    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7g9c6_0e302077-35e3-437a-88ea-a5102042948b/kube-rbac-proxy/0.log"
Jul 18 01:39:13.284543 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:13.284519    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7g9c6_0e302077-35e3-437a-88ea-a5102042948b/exporter/0.log"
Jul 18 01:39:13.305582 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:13.305551    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7g9c6_0e302077-35e3-437a-88ea-a5102042948b/extractor/0.log"
Jul 18 01:39:15.155063 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:15.155038    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dn4z4/perf-node-gather-daemonset-skv45"
Jul 18 01:39:15.513715 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:15.513625    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z8dkt_0972432b-1e47-4eea-9ba5-5d0ab0d94b55/manager/0.log"
Jul 18 01:39:15.771942 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:15.771854    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lkhmr_12dc7c62-a256-4a17-bee1-ed09820e3a01/server/0.log"
Jul 18 01:39:15.855772 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:15.855734    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gpxfc_66288b5c-9cce-42a9-a9ec-7132892da61e/manager/0.log"
Jul 18 01:39:16.003472 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:16.003392    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd8c7c4c4-9n688_3611f732-9623-40c4-bcbe-7bdbd511d053/manager/0.log"
Jul 18 01:39:16.028200 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:16.028108    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd8c7c4c4-9n688_3611f732-9623-40c4-bcbe-7bdbd511d053/copy-manifests/0.log"
Jul 18 01:39:18.497504 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:18.497472    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qgn9p_76d97c25-5c24-4f78-92a8-e1c87e2f0298/istio-proxy/0.log"
Jul 18 01:39:18.638456 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:18.638425    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7569c6bc9b-vmpgv_24c8026f-5014-4d63-95a0-5d46060e2ecc/kube-auth-proxy/0.log"
Jul 18 01:39:18.658171 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:18.658136    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/0.log"
Jul 18 01:39:18.705120 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:18.705085    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/1.log"
Jul 18 01:39:18.761934 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:18.761855    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7df7df96f4-vqk8n_7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608/router/0.log"
Jul 18 01:39:19.965512 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:19.965478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qgn9p_76d97c25-5c24-4f78-92a8-e1c87e2f0298/istio-proxy/0.log"
Jul 18 01:39:20.118860 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:20.118834    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7569c6bc9b-vmpgv_24c8026f-5014-4d63-95a0-5d46060e2ecc/kube-auth-proxy/0.log"
Jul 18 01:39:20.139712 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:20.139680    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/0.log"
Jul 18 01:39:20.183192 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:20.183080    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgrtr_57719e6a-9f81-47e9-a008-69cd60d41d2c/istio-proxy/1.log"
Jul 18 01:39:20.246798 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:20.246677    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7df7df96f4-vqk8n_7bc3c5bf-9d4e-489e-94ae-1ba7fed0a608/router/0.log"
Jul 18 01:39:21.803668 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.803633    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"]
Jul 18 01:39:21.804117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804015    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.804117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804026    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.804117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804032    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.804117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804038    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.804117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804113    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.804117 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804120    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.804439 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.804129    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:21.812484 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.812452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"
Jul 18 01:39:21.814587 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.814558    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"]
Jul 18 01:39:21.814788 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.814764    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 01:39:21.814870 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.814764    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r95h7\""
Jul 18 01:39:21.900203 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:21.900172    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389434f9-2c4e-410c-9417-72374c74b010-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s\" (UID: \"389434f9-2c4e-410c-9417-72374c74b010\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"
Jul 18 01:39:22.002182 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:22.001074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389434f9-2c4e-410c-9417-72374c74b010-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s\" (UID: \"389434f9-2c4e-410c-9417-72374c74b010\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"
Jul 18 01:39:22.002986 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:22.002959    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389434f9-2c4e-410c-9417-72374c74b010-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s\" (UID: \"389434f9-2c4e-410c-9417-72374c74b010\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"
Jul 18 01:39:22.125747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:22.125708    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"
Jul 18 01:39:22.310175 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:22.307843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s"]
Jul 18 01:39:23.141129 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:23.141099    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-klc2f_b36a62f2-c75b-4a7c-97f4-e9dd95ee4254/kube-storage-version-migrator-operator/1.log"
Jul 18 01:39:23.142099 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:23.142079    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-klc2f_b36a62f2-c75b-4a7c-97f4-e9dd95ee4254/kube-storage-version-migrator-operator/0.log"
Jul 18 01:39:23.249684 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:23.249630    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s" event={"ID":"389434f9-2c4e-410c-9417-72374c74b010","Type":"ContainerStarted","Data":"42e201c44c0e3ff485cb7a7748ae055c8293ec6d691870bbee371f63fc45ee6c"}
Jul 18 01:39:23.249873 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:23.249712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s" event={"ID":"389434f9-2c4e-410c-9417-72374c74b010","Type":"ContainerStarted","Data":"0fa8466e04ce99591714d4b0967438887608a9e908745472a868025a3cde61fc"}
Jul 18 01:39:23.264661 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:23.264607    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s" podStartSLOduration=2.264592117 podStartE2EDuration="2.264592117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:39:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:39:23.263587448 +0000 UTC m=+3542.791092410" watchObservedRunningTime="2026-07-18 01:39:23.264592117 +0000 UTC m=+3542.792097076"
Jul 18 01:39:24.246618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.246580    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"]
Jul 18 01:39:24.247015 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.247002    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:24.247080 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.247017    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f82275-fb2c-470a-b4e8-9ef682fe8dfd" containerName="revoke-keys"
Jul 18 01:39:24.256331 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.256301    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"
Jul 18 01:39:24.256908 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.256867    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"]
Jul 18 01:39:24.258571 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.258529    2567 generic.go:356] "Generic (PLEG): container finished" podID="389434f9-2c4e-410c-9417-72374c74b010" containerID="42e201c44c0e3ff485cb7a7748ae055c8293ec6d691870bbee371f63fc45ee6c" exitCode=6
Jul 18 01:39:24.258747 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.258725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s" event={"ID":"389434f9-2c4e-410c-9417-72374c74b010","Type":"ContainerDied","Data":"42e201c44c0e3ff485cb7a7748ae055c8293ec6d691870bbee371f63fc45ee6c"}
Jul 18 01:39:24.259100 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.259064    2567 scope.go:117] "RemoveContainer" containerID="42e201c44c0e3ff485cb7a7748ae055c8293ec6d691870bbee371f63fc45ee6c"
Jul 18 01:39:24.287488 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.287460    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/kube-multus-additional-cni-plugins/0.log"
Jul 18 01:39:24.308927 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.308901    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/egress-router-binary-copy/0.log"
Jul 18 01:39:24.329030 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.328999    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/cni-plugins/0.log"
Jul 18 01:39:24.348900 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.348877    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/bond-cni-plugin/0.log"
Jul 18 01:39:24.369138 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.369110    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/routeoverride-cni/0.log"
Jul 18 01:39:24.391754 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.391732    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/whereabouts-cni-bincopy/0.log"
Jul 18 01:39:24.411442 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.411410    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-smk9b_177e03e0-5d99-449f-aac8-09b9b7f8f6db/whereabouts-cni/0.log"
Jul 18 01:39:24.424308 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.424260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a1f28-33df-4118-95a6-ef2ac164f15d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4hks7\" (UID: \"895a1f28-33df-4118-95a6-ef2ac164f15d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"
Jul 18 01:39:24.525646 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.525558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a1f28-33df-4118-95a6-ef2ac164f15d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4hks7\" (UID: \"895a1f28-33df-4118-95a6-ef2ac164f15d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"
Jul 18 01:39:24.526111 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.526093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a1f28-33df-4118-95a6-ef2ac164f15d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-354f15cb-4hks7\" (UID: \"895a1f28-33df-4118-95a6-ef2ac164f15d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"
Jul 18 01:39:24.569593 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.569565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"
Jul 18 01:39:24.663253 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.663227    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hg9rg_fddc106e-175c-4243-a732-4ebe482246de/kube-multus/0.log"
Jul 18 01:39:24.731650 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.731618    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7"]
Jul 18 01:39:24.735050 ip-10-0-135-55 kubenswrapper[2567]: W0718 01:39:24.735020    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895a1f28_33df_4118_95a6_ef2ac164f15d.slice/crio-1a5cc0bbed7e3dd91af77a484624adf0592e3f7fb2675d853ef263b01c1f089f WatchSource:0}: Error finding container 1a5cc0bbed7e3dd91af77a484624adf0592e3f7fb2675d853ef263b01c1f089f: Status 404 returned error can't find the container with id 1a5cc0bbed7e3dd91af77a484624adf0592e3f7fb2675d853ef263b01c1f089f
Jul 18 01:39:24.802583 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.802552    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qb7rp_984d0621-5433-4613-b1ff-41f803ca0a53/network-metrics-daemon/0.log"
Jul 18 01:39:24.817247 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:24.817218    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qb7rp_984d0621-5433-4613-b1ff-41f803ca0a53/kube-rbac-proxy/0.log"
Jul 18 01:39:25.269843 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.269795    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7" event={"ID":"895a1f28-33df-4118-95a6-ef2ac164f15d","Type":"ContainerStarted","Data":"1c2bdf355f2e282b58c520aac0e54a35c6045465978054cf150e53e5616e7c86"}
Jul 18 01:39:25.270370 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.269867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7" event={"ID":"895a1f28-33df-4118-95a6-ef2ac164f15d","Type":"ContainerStarted","Data":"1a5cc0bbed7e3dd91af77a484624adf0592e3f7fb2675d853ef263b01c1f089f"}
Jul 18 01:39:25.272034 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.271997    2567 generic.go:356] "Generic (PLEG): container finished" podID="389434f9-2c4e-410c-9417-72374c74b010" containerID="79cfa88763bd9bf59e73bc8350a15774791c40d206a25563ed18e9f842e53c72" exitCode=6
Jul 18 01:39:25.272155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.272098    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s" event={"ID":"389434f9-2c4e-410c-9417-72374c74b010","Type":"ContainerDied","Data":"79cfa88763bd9bf59e73bc8350a15774791c40d206a25563ed18e9f842e53c72"}
Jul 18 01:39:25.272155 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.272150    2567 scope.go:117] "RemoveContainer" containerID="42e201c44c0e3ff485cb7a7748ae055c8293ec6d691870bbee371f63fc45ee6c"
Jul 18 01:39:25.272618 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.272596    2567 scope.go:117] "RemoveContainer" containerID="79cfa88763bd9bf59e73bc8350a15774791c40d206a25563ed18e9f842e53c72"
Jul 18 01:39:25.272928 ip-10-0-135-55 kubenswrapper[2567]: E0718 01:39:25.272880    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s_odh-ai-gateway-infra(389434f9-2c4e-410c-9417-72374c74b010)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1b6716ba-gxq9s" podUID="389434f9-2c4e-410c-9417-72374c74b010"
Jul 18 01:39:25.287875 ip-10-0-135-55 kubenswrapper[2567]: I0718 01:39:25.287816    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-354f15cb-4hks7" podStartSLOduration=1.287792777 podStartE2EDuration="1.287792777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:39:25.285527332 +0000 UTC m=+3544.813032293" watchObservedRunningTime="2026-07-18 01:39:25.287792777 +0000 UTC m=+3544.815297740"