Sep 07 11:22:46.791803 ip-10-0-138-45 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 11:22:47.196357 ip-10-0-138-45 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:22:47.196357 ip-10-0-138-45 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 11:22:47.196357 ip-10-0-138-45 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:22:47.196357 ip-10-0-138-45 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 11:22:47.196357 ip-10-0-138-45 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:22:47.197037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.196413    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 11:22:47.199402 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199386    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:22:47.199402 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199402    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199406    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199410    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199413    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199416    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199418    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199421    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199424    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199426    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199428    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199431    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199434    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199437    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199440    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199442    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199445    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199448    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199451    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199454    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:22:47.199463 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199457    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199459    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199463    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199465    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199468    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199471    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199474    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199477    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199480    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199482    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199485    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199499    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199503    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199506    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199508    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199511    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199514    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199516    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199519    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199521    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:22:47.199969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199524    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199526    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199529    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199531    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199534    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199537    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199539    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199542    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199544    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199547    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199550    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199552    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199555    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199558    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199561    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199564    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199567    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199569    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199572    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199574    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:22:47.200451 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199577    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199580    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199582    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199587    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199591    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199595    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199598    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199600    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199603    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199605    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199608    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199610    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199614    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199618    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199621    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199624    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199627    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199630    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199632    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:22:47.200969 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199635    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199637    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199640    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199642    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199645    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199649    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199652    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199657    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199661    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199664    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199667    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199670    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199673    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199676    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199678    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199681    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199683    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199686    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199688    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:22:47.201428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.199691    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200419    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200426    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200429    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200432    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200435    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200438    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200441    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200444    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200446    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200449    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200453    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200456    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200459    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200461    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200464    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200467    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200470    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200473    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200475    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:22:47.201901 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200478    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200481    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200484    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200486    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200503    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200506    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200509    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200511    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200514    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200516    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200519    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200521    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200524    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200527    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200529    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200532    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200534    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200537    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200541    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200545    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:22:47.202427 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200548    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200551    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200554    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200557    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200560    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200564    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200566    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200569    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200573    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200576    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200580    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200583    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200586    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200589    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200591    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200595    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200597    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200600    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200602    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200605    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:22:47.202925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200607    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200610    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200612    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200615    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200617    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200619    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200622    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200624    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200627    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200630    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200632    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200635    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200638    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200640    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200642    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200645    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200648    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200651    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200653    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200656    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:22:47.203414 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200658    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200661    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200665    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200668    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200671    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200673    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200676    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200678    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200681    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200683    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200686    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200688    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200690    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200693    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200696    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200698    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200703    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200706    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200709    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:22:47.203911 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.200712    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201080    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201086    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201088    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201091    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201094    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201097    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201099    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201102    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201106    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201108    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201112    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201116    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201128    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201132    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201135    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201137    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201141    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201144    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:22:47.204358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201147    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201150    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201152    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201155    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201158    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201160    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201162    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201165    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201167    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201171    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201175    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201178    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201180    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201183    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201186    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201189    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201191    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201195    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201197    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:22:47.204848 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201200    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201202    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201205    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201208    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201211    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201214    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201217    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201219    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201223    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201226    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201229    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201232    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201234    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201237    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201239    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201241    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201244    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201246    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201249    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201251    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:22:47.205313 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201254    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201256    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201259    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201261    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201264    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201266    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201268    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201271    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201274    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201277    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201279    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201282    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201285    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201288    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201290    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201293    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201295    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201298    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201301    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201304    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:22:47.205818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201306    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201308    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201311    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201314    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201316    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201319    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201321    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201324    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201326    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201329    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201331    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201334    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201336    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201339    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201341    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201344    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201346    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201349    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201354    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201356    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:22:47.206337 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201359    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.201362    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201449    2560 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201509    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201516    2560 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201520    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201525    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201528    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201534    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201538    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201542    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201545    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201549    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201552    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201555    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201558    2560 flags.go:64] FLAG: --cgroup-root=""
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201562    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201565    2560 flags.go:64] FLAG: --client-ca-file=""
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201568    2560 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201571    2560 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201576    2560 flags.go:64] FLAG: --cluster-domain=""
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201579    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201582    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201585    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 11:22:47.206832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201589    2560 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201593    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201596    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201599    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201603    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201607    2560 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201611    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201614    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201618    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201621    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201625    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201628    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201631    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201634    2560 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201637    2560 flags.go:64] FLAG: --enable-server="true"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.201643    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202115    2560 flags.go:64] FLAG: --event-burst="100"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202119    2560 flags.go:64] FLAG: --event-qps="50"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202122    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202126    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202130    2560 flags.go:64] FLAG: --eviction-hard=""
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202134    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202137    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202140    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202143    2560 flags.go:64] FLAG: --eviction-soft=""
Sep 07 11:22:47.207485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202146    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202149    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202152    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202154    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202157    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202160    2560 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202163    2560 flags.go:64] FLAG: --feature-gates=""
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202167    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202170    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202173    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202176    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202178    2560 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202182    2560 flags.go:64] FLAG: --help="false"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202185    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202188    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202191    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202194    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202198    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202201    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202204    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202207    2560 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202210    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202214    2560 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 11:22:47.208131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202217    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202220    2560 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202223    2560 flags.go:64] FLAG: --kube-reserved=""
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202225    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202229    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202233    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202236    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202238    2560 flags.go:64] FLAG: --lock-file=""
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202242    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202244    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202247    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202253    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202255    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202259    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202261    2560 flags.go:64] FLAG: --logging-format="text"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202264    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202267    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202270    2560 flags.go:64] FLAG: --manifest-url=""
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202273    2560 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202277    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202280    2560 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202284    2560 flags.go:64] FLAG: --max-pods="110"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202287    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202291    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202293    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 11:22:47.208750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202296    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202299    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202302    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202305    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202312    2560 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202316    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202320    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202323    2560 flags.go:64] FLAG: --pod-cidr=""
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202326    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202331    2560 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202334    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202338    2560 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202347    2560 flags.go:64] FLAG: --port="10250"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202350    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202353    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08d42a425a85c15cb"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202356    2560 flags.go:64] FLAG: --qos-reserved=""
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202359    2560 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202362    2560 flags.go:64] FLAG: --register-node="true"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202365    2560 flags.go:64] FLAG: --register-with-taints=""
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202369    2560 flags.go:64] FLAG: --registry-burst="10"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202372    2560 flags.go:64] FLAG: --registry-qps="5"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202375    2560 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202378    2560 flags.go:64] FLAG: --reserved-memory=""
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202381    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202384    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 11:22:47.209396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202388    2560 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202391    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202393    2560 flags.go:64] FLAG: --runonce="false"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202396    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202399    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202402    2560 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202405    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202408    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202411    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202414    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202417    2560 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202420    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202423    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202427    2560 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202430    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202433    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202436    2560 flags.go:64] FLAG: --system-cgroups=""
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202439    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202444    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202447    2560 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202450    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202455    2560 flags.go:64] FLAG: --tls-min-version=""
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202458    2560 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202461    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202464    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 11:22:47.210034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202466    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202469    2560 flags.go:64] FLAG: --v="2"
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202473    2560 flags.go:64] FLAG: --version="false"
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202477    2560 flags.go:64] FLAG: --vmodule=""
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202481    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.202484    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202616    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202620    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202623    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202626    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202629    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202632    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202634    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202637    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202640    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202644    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202647    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202651    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202653    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202656    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202659    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:22:47.210667 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202662    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202665    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202667    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202670    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202672    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202675    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202677    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202680    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202683    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202686    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202688    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202691    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202693    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202696    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202698    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202701    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202703    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202706    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202708    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202711    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:22:47.211165 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202713    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202717    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202720    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202723    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202725    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202728    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202730    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202733    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202736    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202738    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202741    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202744    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202747    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202750    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202752    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202755    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202757    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202760    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202762    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:22:47.211703 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202765    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202768    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202770    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202773    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202775    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202778    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202780    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202783    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202786    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202788    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202791    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202793    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202796    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202798    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202803    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202805    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202808    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202810    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202813    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202816    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:22:47.212160 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202818    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202821    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202823    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202826    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202830    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202833    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202836    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202840    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202843    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202846    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202849    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202852    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202855    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202858    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202862    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202865    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202868    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202870    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202874    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:22:47.212652 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202876    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202879    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202881    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202884    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202886    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.202889    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.203459    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.211250    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.211288    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.212024    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.212036    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212108    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212116    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212120    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212123    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:22:47.213121 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212126    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212129    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212131    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212134    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212136    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212140    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212142    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212145    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212147    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212150    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212154    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212158    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212161    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212164    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212167    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212170    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212172    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212175    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212178    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:22:47.213486 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212183    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212187    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212191    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212196    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212199    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212201    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212211    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212214    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212217    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212219    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212222    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212224    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212227    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212230    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212234    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212237    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212241    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212244    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212247    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:22:47.213968 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212250    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212252    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212255    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212259    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212264    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212268    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212272    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212276    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212279    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212282    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212284    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212288    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212291    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212293    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212296    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212299    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212301    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212304    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212306    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:22:47.214452 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212309    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212318    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212321    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212323    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212326    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212328    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212331    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212334    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212336    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212340    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212344    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212348    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212352    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212356    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212359    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212361    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212364    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212366    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212369    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212372    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:22:47.214967 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212374    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212377    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212380    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212382    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212384    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212387    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212389    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212392    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212394    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212397    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212400    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212402    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212404    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212407    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212418    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212422    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212426    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:22:47.215449 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212431    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.212438    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212619    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212628    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212631    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212634    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212636    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212639    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212642    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212644    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212646    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212649    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212652    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212654    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212657    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:22:47.215888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212661    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212666    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212670    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212674    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212678    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212681    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212684    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212686    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212689    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212693    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212698    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212701    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212704    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212707    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212710    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212718    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212721    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212724    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212726    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:22:47.216246 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212729    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212731    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212734    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212737    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212739    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212744    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212748    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212752    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212756    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212760    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212762    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212772    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212775    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212778    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212780    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212783    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212785    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212788    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212790    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212793    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:22:47.216723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212795    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212798    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212801    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212803    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212806    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212809    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212811    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212814    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212816    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212827    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212832    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212837    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212840    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212843    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212845    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212848    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212850    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212853    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212855    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212858    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:22:47.217216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212860    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212864    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212868    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212871    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212874    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212877    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212879    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212882    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212884    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212887    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212890    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212893    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212895    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212898    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212902    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212906    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212910    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212914    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212918    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:22:47.217711 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212921    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212923    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212925    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212934    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212937    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212939    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212942    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:47.212944    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.212949    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.213091    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.215218    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.216179    2560 server.go:1019] "Starting client certificate rotation"
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.216275    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 11:22:47.218410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.216306    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 11:22:47.221715 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.221697    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 11:22:47.223638 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.223616    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 11:22:47.232625 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.232610    2560 log.go:25] "Validated CRI v1 runtime API"
Sep 07 11:22:47.237401 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.237384    2560 log.go:25] "Validated CRI v1 image API"
Sep 07 11:22:47.238324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.238308    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 11:22:47.240581 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.240558    2560 fs.go:135] Filesystem UUIDs: map[38861d16-6916-46a2-8479-7895e45eba81:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e8a96fea-f8a8-4de4-b398-4bbeba3a9e01:/dev/nvme0n1p4]
Sep 07 11:22:47.240637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.240581    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 11:22:47.246659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.246638    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 11:22:47.248603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.248476    2560 manager.go:216] Machine: {Timestamp:2026-09-07 11:22:47.245262166 +0000 UTC m=+0.338862098 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102770 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e0a13744792fddf1a1483d00e5bf7 SystemUUID:ec2e0a13-7447-92fd-df1a-1483d00e5bf7 BootID:2dae4789-d6e5-4fd7-b833-2f590cf98d5d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:4f:8e:3c:8b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:4f:8e:3c:8b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:75:28:7d:32:a0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 11:22:47.248603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.248595    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 11:22:47.248730 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.248684    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 11:22:47.251625 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.251599    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 11:22:47.251763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.251627    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-45.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 11:22:47.252577 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.252567    2560 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 11:22:47.252616 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.252580    2560 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 11:22:47.252616 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.252588    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 11:22:47.253326 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.253315    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 11:22:47.254693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.254673    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 11:22:47.257821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257809    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 11:22:47.257933 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257924    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 11:22:47.257966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257952    2560 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 11:22:47.257966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257962    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 11:22:47.258018 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257975    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 11:22:47.258018 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257986    2560 kubelet.go:398] "Adding apiserver pod source"
Sep 07 11:22:47.258018 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.257995    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 11:22:47.258439 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.258426    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 11:22:47.258503 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.258445    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 11:22:47.260831 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.260814    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 11:22:47.263187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263171    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 11:22:47.263245 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263213    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 11:22:47.263396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263383    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 11:22:47.263442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263406    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 11:22:47.263442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263415    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 11:22:47.263442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263421    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 11:22:47.263442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263430    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 11:22:47.263442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263435    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 11:22:47.263891 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263882    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 11:22:47.263921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263891    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 11:22:47.263921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263898    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 11:22:47.263921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263904    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 11:22:47.263921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263913    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 11:22:47.263921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263921    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 11:22:47.264048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263945    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 11:22:47.264048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.263954    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 11:22:47.269125 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.269102    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-45.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 11:22:47.269762 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.269744    2560 server.go:1268] "Started kubelet"
Sep 07 11:22:47.269880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.269820    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 11:22:47.269935 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.269870    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 11:22:47.269985 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.269948    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 11:22:47.270742 ip-10-0-138-45 systemd[1]: Started Kubernetes Kubelet.
Sep 07 11:22:47.271232 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.271203    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-45.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 11:22:47.271329 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.271227    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 11:22:47.271394 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.271361    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 11:22:47.271670 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.271656    2560 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 11:22:47.273599 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.273574    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fhsts"
Sep 07 11:22:47.275981 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.275963    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 11:22:47.276064 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.275968    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 11:22:47.276120 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.276075    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.276644 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.276623    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 11:22:47.276837 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.275621    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-45.ec2.internal.18d30615b35ff269  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-45.ec2.internal,UID:ip-10-0-138-45.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-45.ec2.internal,},FirstTimestamp:2026-09-07 11:22:47.269708393 +0000 UTC m=+0.363308321,LastTimestamp:2026-09-07 11:22:47.269708393 +0000 UTC m=+0.363308321,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-45.ec2.internal,}"
Sep 07 11:22:47.276937 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.276882    2560 factory.go:55] Registering systemd factory
Sep 07 11:22:47.276937 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.276898    2560 factory.go:223] Registration of the systemd container factory successfully
Sep 07 11:22:47.277114 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277086    2560 factory.go:153] Registering CRI-O factory
Sep 07 11:22:47.277114 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277113    2560 factory.go:223] Registration of the crio container factory successfully
Sep 07 11:22:47.277247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277159    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 11:22:47.277247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277179    2560 factory.go:103] Registering Raw factory
Sep 07 11:22:47.277247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277196    2560 manager.go:1196] Started watching for new ooms in manager
Sep 07 11:22:47.277592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277575    2560 manager.go:349] Starting recovery of all containers
Sep 07 11:22:47.277829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277812    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 11:22:47.277829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277831    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 11:22:47.277941 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277898    2560 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 11:22:47.277941 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.277910    2560 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 11:22:47.280792 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.280759    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-45.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 07 11:22:47.280908 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.280847    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 07 11:22:47.284725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.284703    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fhsts"
Sep 07 11:22:47.289541 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.289519    2560 manager.go:354] Recovery completed
Sep 07 11:22:47.294083 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.294067    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.295967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.295855    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.296040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.295979    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.296040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.295990    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.296465 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.296449    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 11:22:47.296465 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.296463    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 11:22:47.296603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.296471    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.296603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.296477    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 11:22:47.297205 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.297188    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.297253 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.297221    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.297253 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.297237    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.297763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.297752    2560 policy_none.go:49] "None policy: Start"
Sep 07 11:22:47.297815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.297766    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 11:22:47.297815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.297777    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 11:22:47.298074 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.297989    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-45.ec2.internal.18d30615b4f09af2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-45.ec2.internal,UID:ip-10-0-138-45.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-45.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-45.ec2.internal,},FirstTimestamp:2026-09-07 11:22:47.295965938 +0000 UTC m=+0.389565865,LastTimestamp:2026-09-07 11:22:47.295965938 +0000 UTC m=+0.389565865,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-45.ec2.internal,}"
Sep 07 11:22:47.299177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.299165    2560 policy_none.go:47] "Start"
Sep 07 11:22:47.328132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328116    2560 manager.go:337] "Starting Device Plugin manager"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.328152    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328166    2560 server.go:87] "Starting device plugin registration server"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328572    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328591    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328734    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328837    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.328847    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.329393    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.329456    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.343309    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.344409    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.344450    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.344470    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.344569    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 11:22:47.348873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.346636    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:22:47.429747 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.429722    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.430750 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.430729    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.430853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.430766    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.430853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.430781    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.430853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.430818    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.439149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.439132    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.439252 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.439156    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-45.ec2.internal\": node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.444675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.444654    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"]
Sep 07 11:22:47.444760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.444720    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.445511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.445480    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.445586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.445526    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.445586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.445540    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.447011 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.446949    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.447074 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447058    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.447120 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447084    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.447650 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447632    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.447650 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447642    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.447777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447663    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.447777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447667    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.447777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447674    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.447777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.447681    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.449234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.449209    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.449234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.449237    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:22:47.449886 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.449866    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:22:47.449991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.449894    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:22:47.449991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.449905    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:22:47.451188 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.451174    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.479189 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.479163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00b2bc161ab733dadf328ac57d8c9e64-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal\" (UID: \"00b2bc161ab733dadf328ac57d8c9e64\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.479189 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.479190    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00b2bc161ab733dadf328ac57d8c9e64-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal\" (UID: \"00b2bc161ab733dadf328ac57d8c9e64\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.479332 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.479217    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/422a2dc891c89e8aeb801cae87a1e08b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.ec2.internal\" (UID: \"422a2dc891c89e8aeb801cae87a1e08b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.487291 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.487272    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-45.ec2.internal\" not found" node="ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.491707 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.491690    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-45.ec2.internal\" not found" node="ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.552069 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.552042    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.580263 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.580218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00b2bc161ab733dadf328ac57d8c9e64-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal\" (UID: \"00b2bc161ab733dadf328ac57d8c9e64\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.580343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.580197    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00b2bc161ab733dadf328ac57d8c9e64-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal\" (UID: \"00b2bc161ab733dadf328ac57d8c9e64\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.580343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.580314    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00b2bc161ab733dadf328ac57d8c9e64-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal\" (UID: \"00b2bc161ab733dadf328ac57d8c9e64\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.580343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.580335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/422a2dc891c89e8aeb801cae87a1e08b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.ec2.internal\" (UID: \"422a2dc891c89e8aeb801cae87a1e08b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.580442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.580351    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00b2bc161ab733dadf328ac57d8c9e64-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal\" (UID: \"00b2bc161ab733dadf328ac57d8c9e64\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.580442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.580363    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/422a2dc891c89e8aeb801cae87a1e08b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-45.ec2.internal\" (UID: \"422a2dc891c89e8aeb801cae87a1e08b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.652376 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.652345    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.753073 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.752994    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.790661 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.790630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.794833 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:47.794812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:47.854061 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.854024    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:47.954626 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:47.954594    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:48.055122 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:48.055035    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:48.155581 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:48.155548    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:48.174762 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:48.174735    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:22:48.216568 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:48.216542    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 11:22:48.217011 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:48.216683    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:22:48.217011 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:48.216683    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:22:49.402104 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:49.402069    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:49.435767 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.435712    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 11:17:48 +0000 UTC" deadline="2028-02-16 04:50:46.47934025 +0000 UTC"
Sep 07 11:22:49.435863 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.435767    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12641h27m57.043577789s"
Sep 07 11:22:49.438443 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:49.438413    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422a2dc891c89e8aeb801cae87a1e08b.slice/crio-b5f213ef16c68ab57b8d2ecd195e00f95e30d3cefd0771fe9cbb7a65690a8f41 WatchSource:0}: Error finding container b5f213ef16c68ab57b8d2ecd195e00f95e30d3cefd0771fe9cbb7a65690a8f41: Status 404 returned error can't find the container with id b5f213ef16c68ab57b8d2ecd195e00f95e30d3cefd0771fe9cbb7a65690a8f41
Sep 07 11:22:49.438807 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:49.438787    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b2bc161ab733dadf328ac57d8c9e64.slice/crio-f499115447c7c127e8ad37fa77d7d65e413214faf77273c2aef6d4862813cd61 WatchSource:0}: Error finding container f499115447c7c127e8ad37fa77d7d65e413214faf77273c2aef6d4862813cd61: Status 404 returned error can't find the container with id f499115447c7c127e8ad37fa77d7d65e413214faf77273c2aef6d4862813cd61
Sep 07 11:22:49.444647 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.444623    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:22:49.494366 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.494247    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal" event={"ID":"00b2bc161ab733dadf328ac57d8c9e64","Type":"ContainerStarted","Data":"f499115447c7c127e8ad37fa77d7d65e413214faf77273c2aef6d4862813cd61"}
Sep 07 11:22:49.495412 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.495394    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal" event={"ID":"422a2dc891c89e8aeb801cae87a1e08b","Type":"ContainerStarted","Data":"b5f213ef16c68ab57b8d2ecd195e00f95e30d3cefd0771fe9cbb7a65690a8f41"}
Sep 07 11:22:49.502562 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:49.502545    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:49.603092 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:49.603057    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:49.703607 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:49.703575    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:49.775098 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.775019    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:22:49.803765 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:49.803718    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:49.904678 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:49.904640    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-45.ec2.internal\" not found"
Sep 07 11:22:49.907373 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.907345    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:22:49.923075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.923048    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:49.933929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.933795    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 11:22:49.935029 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.934836    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal"
Sep 07 11:22:49.943008 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:49.942977    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 11:22:50.405368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.405334    2560 apiserver.go:52] "Watching apiserver"
Sep 07 11:22:50.412129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.412103    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 11:22:50.412506 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.412481    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-bjfn8","openshift-ovn-kubernetes/ovnkube-node-z8clx","kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal","openshift-cluster-node-tuning-operator/tuned-m7hfb","openshift-multus/multus-additional-cni-plugins-zrp82","openshift-multus/multus-lhhhk","openshift-network-diagnostics/network-check-target-6628l","openshift-network-operator/iptables-alerter-mvf88","kube-system/global-pull-secret-syncer-6dsdg","kube-system/konnectivity-agent-lwmzh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts","openshift-dns/node-resolver-fjx89","openshift-image-registry/node-ca-q4gvm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal"]
Sep 07 11:22:50.414212 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.414192    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:50.414337 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.414263    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:22:50.415508 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.415482    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.416947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.416925    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.417528 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.417507    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 11:22:50.417641 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.417617    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.417858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.417843    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.417953 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.417931    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9frs\""
Sep 07 11:22:50.418275 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.418258    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.419064 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.419034    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.419064 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.419047    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j27fp\""
Sep 07 11:22:50.419209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.419123    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 11:22:50.419290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.419263    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.419631 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.419615    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.420166 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.420144    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.420417 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.420393    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h8gjf\""
Sep 07 11:22:50.420417 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.420402    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.421162 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.421145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.422345 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422326    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 11:22:50.422442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422331    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.422442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422378    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 11:22:50.422442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422429    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.422822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422804    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bv9nt\""
Sep 07 11:22:50.422927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422910    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 11:22:50.422986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.422969    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 11:22:50.423204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.423189    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.423279 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.423254    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:22:50.423548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.423528    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 11:22:50.423627 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.423556    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rdtd4\""
Sep 07 11:22:50.424555 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.424529    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:50.424647 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.424596    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:22:50.426204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.426186    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.428524 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.428384    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.428524 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.428419    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 11:22:50.428524 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.428428    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 11:22:50.430916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.430884    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.432549 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.432519    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.433403 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.433379    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 11:22:50.433988 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.433966    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8kv7f\""
Sep 07 11:22:50.434248 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.434225    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 11:22:50.435692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-etc-kubernetes\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.435812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62b756ad-9c5d-493f-9afa-553c4a741503-dbus\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.435812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435767    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:50.435812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435801    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nllp\" (UniqueName: \"kubernetes.io/projected/9e39c68a-5691-47a5-bb3c-fa289e615054-kube-api-access-6nllp\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:50.435987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435838    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-cnibin\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.435987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-socket-dir-parent\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.435995    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.436144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436075    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfvqp\" (UniqueName: \"kubernetes.io/projected/4f8d8134-6363-4b59-81c9-9b01c1b2df54-kube-api-access-pfvqp\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.436144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-iptables-alerter-script\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.436326 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-systemd\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.436326 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.436326 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f5ad469-1623-4329-a48d-e71071315cdd-cni-binary-copy\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436326 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436314    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-k8s-cni-cncf-io\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4f5ad469-1623-4329-a48d-e71071315cdd-multus-daemon-config\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysconfig\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.436491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436432    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysctl-d\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.436491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-cni-binary-copy\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.436706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436530    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlxb9\" (UniqueName: \"kubernetes.io/projected/e2760e4f-b854-488a-9886-e92ade779208-kube-api-access-rlxb9\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.436706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-multus-certs\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-var-lib-kubelet\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.436706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436635    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.436943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-host\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.436943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436773    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-system-cni-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-cnibin\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.436943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436876    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 11:22:50.437210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.436876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-cni-bin\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.437259 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62b756ad-9c5d-493f-9afa-553c4a741503-kubelet-config\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.437292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f8d8134-6363-4b59-81c9-9b01c1b2df54-host\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.437292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f8d8134-6363-4b59-81c9-9b01c1b2df54-serviceca\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.437353 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-run\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.437353 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437306    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t2m96\""
Sep 07 11:22:50.437353 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437316    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-sys\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.437353 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-tuned\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.437462 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b017d57-3f8c-44a8-b277-b723bb14b44a-tmp\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.437462 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437395    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:50.437462 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-host-slash\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.437624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqt95\" (UniqueName: \"kubernetes.io/projected/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-kube-api-access-qqt95\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.437624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysctl-conf\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.437624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437601    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.437826 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.437653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-netns\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-kubelet\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438300    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 11:22:50.438343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-hostroot\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttnnq\" (UniqueName: \"kubernetes.io/projected/2b017d57-3f8c-44a8-b277-b723bb14b44a-kube-api-access-ttnnq\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-system-cni-dir\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-cni-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438493    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-cni-multus\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438513    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gtd26\""
Sep 07 11:22:50.438539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-conf-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438543    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fng4q\" (UniqueName: \"kubernetes.io/projected/4f5ad469-1623-4329-a48d-e71071315cdd-kube-api-access-fng4q\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.438909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438567    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-modprobe-d\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.438909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-kubernetes\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.438909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438609    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-lib-modules\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.438909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.438640    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.439213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.439106    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 11:22:50.439283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.439225    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.439403 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.439384    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.440424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.440407    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 11:22:50.440731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.440709    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 11:22:50.440836 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.440714    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wqspj\""
Sep 07 11:22:50.443328 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.443311    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 11:22:50.459222 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.459202    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vvnb5"
Sep 07 11:22:50.466999 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.466975    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vvnb5"
Sep 07 11:22:50.523946 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.523920    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 11:22:50.539141 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539107    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-cni-binary-copy\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.539324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539164    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlxb9\" (UniqueName: \"kubernetes.io/projected/e2760e4f-b854-488a-9886-e92ade779208-kube-api-access-rlxb9\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.539324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.539324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539226    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c0a647-41c3-4e09-8060-ad1ae7940794-tmp-dir\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.539324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-var-lib-kubelet\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.539324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-host\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-host\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-var-lib-kubelet\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539421    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-etc-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-netns\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539479    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-hostroot\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-etc-kubernetes\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539512    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-netns\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62b756ad-9c5d-493f-9afa-553c4a741503-dbus\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539545    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-hostroot\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-slash\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539569    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-etc-kubernetes\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f8d8134-6363-4b59-81c9-9b01c1b2df54-host\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysctl-conf\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539615    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62b756ad-9c5d-493f-9afa-553c4a741503-dbus\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.539615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-run\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f8d8134-6363-4b59-81c9-9b01c1b2df54-host\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b017d57-3f8c-44a8-b277-b723bb14b44a-tmp\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-var-lib-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.539710    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-cni-bin\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539755    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-cni-binary-copy\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysctl-conf\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-ovnkube-config\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d3cbf66-5aad-433f-9f7a-79dc9c747517-agent-certs\") pod \"konnectivity-agent-lwmzh\" (UID: \"8d3cbf66-5aad-433f-9f7a-79dc9c747517\") " pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539890    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-run\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.539901    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:51.039854215 +0000 UTC m=+2.987204980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttnnq\" (UniqueName: \"kubernetes.io/projected/2b017d57-3f8c-44a8-b277-b723bb14b44a-kube-api-access-ttnnq\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539972    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-system-cni-dir\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.539996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-cni-multus\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fng4q\" (UniqueName: \"kubernetes.io/projected/4f5ad469-1623-4329-a48d-e71071315cdd-kube-api-access-fng4q\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-system-cni-dir\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.540359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-cni-multus\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540047    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-run-netns\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d3cbf66-5aad-433f-9f7a-79dc9c747517-konnectivity-ca\") pod \"konnectivity-agent-lwmzh\" (UID: \"8d3cbf66-5aad-433f-9f7a-79dc9c747517\") " pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6dwt\" (UniqueName: \"kubernetes.io/projected/796c1991-a31c-45ad-b16c-e3d0eeef05e1-kube-api-access-s6dwt\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-modprobe-d\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-lib-modules\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-systemd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540240    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-modprobe-d\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540267    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540323    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc4k7\" (UniqueName: \"kubernetes.io/projected/d0c0a647-41c3-4e09-8060-ad1ae7940794-kube-api-access-wc4k7\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-registration-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540370    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540377    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-lib-modules\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0c0a647-41c3-4e09-8060-ad1ae7940794-hosts-file\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540437    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfvqp\" (UniqueName: \"kubernetes.io/projected/4f8d8134-6363-4b59-81c9-9b01c1b2df54-kube-api-access-pfvqp\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f5ad469-1623-4329-a48d-e71071315cdd-cni-binary-copy\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4f5ad469-1623-4329-a48d-e71071315cdd-multus-daemon-config\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540631    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-device-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysconfig\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-multus-certs\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-system-cni-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540733    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-cnibin\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-cni-bin\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62b756ad-9c5d-493f-9afa-553c4a741503-kubelet-config\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540826    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540851    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nllp\" (UniqueName: \"kubernetes.io/projected/9e39c68a-5691-47a5-bb3c-fa289e615054-kube-api-access-6nllp\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540877    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f8d8134-6363-4b59-81c9-9b01c1b2df54-serviceca\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-sys\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-tuned\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.540998    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysconfig\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-host-slash\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541022    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4f5ad469-1623-4329-a48d-e71071315cdd-multus-daemon-config\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.541605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541029    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f5ad469-1623-4329-a48d-e71071315cdd-cni-binary-copy\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541043    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqt95\" (UniqueName: \"kubernetes.io/projected/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-kube-api-access-qqt95\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541052    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-multus-certs\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541073    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-kubelet\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-sys\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-kubelet\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541234    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-system-cni-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541282    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62b756ad-9c5d-493f-9afa-553c4a741503-kubelet-config\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-var-lib-cni-bin\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541351    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-env-overrides\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f8d8134-6363-4b59-81c9-9b01c1b2df54-serviceca\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541437    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-cni-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-host-slash\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-conf-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-kubelet\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-systemd-units\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.542308 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.541511    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.541586    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:51.041569683 +0000 UTC m=+2.988920459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-ovn\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541688    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjdpd\" (UniqueName: \"kubernetes.io/projected/a05961a1-9a71-424f-b728-7764722ae488-kube-api-access-kjdpd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541757    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-kubernetes\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541774    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-cni-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-conf-dir\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541864    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-ovnkube-script-lib\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-sys-fs\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-cnibin\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541949    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-kubernetes\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.541949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-socket-dir-parent\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542000    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-node-log\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542007    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-cnibin\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-log-socket\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2760e4f-b854-488a-9886-e92ade779208-cnibin\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-multus-socket-dir-parent\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542084    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-cni-netd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542144    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-socket-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-iptables-alerter-script\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-systemd\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-systemd\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542344    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542702    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-iptables-alerter-script\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-k8s-cni-cncf-io\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542803    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a05961a1-9a71-424f-b728-7764722ae488-ovn-node-metrics-cert\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysctl-d\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2760e4f-b854-488a-9886-e92ade779208-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4f5ad469-1623-4329-a48d-e71071315cdd-host-run-k8s-cni-cncf-io\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.543870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.542935    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-sysctl-d\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.544704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.544232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b017d57-3f8c-44a8-b277-b723bb14b44a-etc-tuned\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.544704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.544284    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b017d57-3f8c-44a8-b277-b723bb14b44a-tmp\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.546624 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.546606    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:22:50.546624 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.546628    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:22:50.546819 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.546643    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:50.546819 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:50.546707    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:51.046688794 +0000 UTC m=+2.994039559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:50.548180 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.548114    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlxb9\" (UniqueName: \"kubernetes.io/projected/e2760e4f-b854-488a-9886-e92ade779208-kube-api-access-rlxb9\") pod \"multus-additional-cni-plugins-zrp82\" (UID: \"e2760e4f-b854-488a-9886-e92ade779208\") " pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.548428 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.548406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fng4q\" (UniqueName: \"kubernetes.io/projected/4f5ad469-1623-4329-a48d-e71071315cdd-kube-api-access-fng4q\") pod \"multus-lhhhk\" (UID: \"4f5ad469-1623-4329-a48d-e71071315cdd\") " pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.549161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.549117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttnnq\" (UniqueName: \"kubernetes.io/projected/2b017d57-3f8c-44a8-b277-b723bb14b44a-kube-api-access-ttnnq\") pod \"tuned-m7hfb\" (UID: \"2b017d57-3f8c-44a8-b277-b723bb14b44a\") " pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.549639 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.549616    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqt95\" (UniqueName: \"kubernetes.io/projected/2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4-kube-api-access-qqt95\") pod \"iptables-alerter-mvf88\" (UID: \"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4\") " pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.550703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.550677    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nllp\" (UniqueName: \"kubernetes.io/projected/9e39c68a-5691-47a5-bb3c-fa289e615054-kube-api-access-6nllp\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:50.551934 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.551912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfvqp\" (UniqueName: \"kubernetes.io/projected/4f8d8134-6363-4b59-81c9-9b01c1b2df54-kube-api-access-pfvqp\") pod \"node-ca-q4gvm\" (UID: \"4f8d8134-6363-4b59-81c9-9b01c1b2df54\") " pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.643898 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.643856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-registration-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.643921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0c0a647-41c3-4e09-8060-ad1ae7940794-hosts-file\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.643946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-device-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.643982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-env-overrides\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.643979    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-registration-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.643999    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-kubelet\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0c0a647-41c3-4e09-8060-ad1ae7940794-hosts-file\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.644077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644038    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-systemd-units\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-ovn\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644111    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjdpd\" (UniqueName: \"kubernetes.io/projected/a05961a1-9a71-424f-b728-7764722ae488-kube-api-access-kjdpd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644128    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-device-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644067    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-systemd-units\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644153    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-kubelet\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644194    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644200    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-ovn\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-ovnkube-script-lib\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-sys-fs\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-node-log\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-log-socket\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644354    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-cni-netd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-sys-fs\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.644413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644386    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-node-log\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-socket-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644457    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-cni-netd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644479    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-env-overrides\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-log-socket\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a05961a1-9a71-424f-b728-7764722ae488-ovn-node-metrics-cert\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644562    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c0a647-41c3-4e09-8060-ad1ae7940794-tmp-dir\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-etc-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-socket-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644602    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/796c1991-a31c-45ad-b16c-e3d0eeef05e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644619    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-slash\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644650    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-etc-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644655    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-var-lib-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644678    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-var-lib-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-cni-bin\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-cni-bin\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644705    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-slash\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-ovnkube-config\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d3cbf66-5aad-433f-9f7a-79dc9c747517-agent-certs\") pod \"konnectivity-agent-lwmzh\" (UID: \"8d3cbf66-5aad-433f-9f7a-79dc9c747517\") " pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644826    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-run-netns\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d3cbf66-5aad-433f-9f7a-79dc9c747517-konnectivity-ca\") pod \"konnectivity-agent-lwmzh\" (UID: \"8d3cbf66-5aad-433f-9f7a-79dc9c747517\") " pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6dwt\" (UniqueName: \"kubernetes.io/projected/796c1991-a31c-45ad-b16c-e3d0eeef05e1-kube-api-access-s6dwt\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644880    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-ovnkube-script-lib\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-run-netns\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c0a647-41c3-4e09-8060-ad1ae7940794-tmp-dir\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-systemd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.644976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc4k7\" (UniqueName: \"kubernetes.io/projected/d0c0a647-41c3-4e09-8060-ad1ae7940794-kube-api-access-wc4k7\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645181    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-openvswitch\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-run-systemd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05961a1-9a71-424f-b728-7764722ae488-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645280    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a05961a1-9a71-424f-b728-7764722ae488-ovnkube-config\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.645853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.645347    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d3cbf66-5aad-433f-9f7a-79dc9c747517-konnectivity-ca\") pod \"konnectivity-agent-lwmzh\" (UID: \"8d3cbf66-5aad-433f-9f7a-79dc9c747517\") " pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.646819 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.646799    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a05961a1-9a71-424f-b728-7764722ae488-ovn-node-metrics-cert\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.647003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.646973    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d3cbf66-5aad-433f-9f7a-79dc9c747517-agent-certs\") pod \"konnectivity-agent-lwmzh\" (UID: \"8d3cbf66-5aad-433f-9f7a-79dc9c747517\") " pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.654827 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.654806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6dwt\" (UniqueName: \"kubernetes.io/projected/796c1991-a31c-45ad-b16c-e3d0eeef05e1-kube-api-access-s6dwt\") pod \"aws-ebs-csi-driver-node-z26ts\" (UID: \"796c1991-a31c-45ad-b16c-e3d0eeef05e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.655467 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.655419    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjdpd\" (UniqueName: \"kubernetes.io/projected/a05961a1-9a71-424f-b728-7764722ae488-kube-api-access-kjdpd\") pod \"ovnkube-node-z8clx\" (UID: \"a05961a1-9a71-424f-b728-7764722ae488\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.657246 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.657228    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc4k7\" (UniqueName: \"kubernetes.io/projected/d0c0a647-41c3-4e09-8060-ad1ae7940794-kube-api-access-wc4k7\") pod \"node-resolver-fjx89\" (UID: \"d0c0a647-41c3-4e09-8060-ad1ae7940794\") " pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:50.723024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.722984    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:22:50.727588 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.727560    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q4gvm"
Sep 07 11:22:50.736726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.736704    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mvf88"
Sep 07 11:22:50.746584 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.746548    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7hfb"
Sep 07 11:22:50.751481 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.751460    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zrp82"
Sep 07 11:22:50.758719 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.758700    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lhhhk"
Sep 07 11:22:50.765653 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.765636    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:22:50.773671 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.773651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:22:50.780434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.780413    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts"
Sep 07 11:22:50.787240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:50.787222    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fjx89"
Sep 07 11:22:51.047627 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.047549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:51.047627 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.047598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.047651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047713    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047771    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047793    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047818    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047833    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:51.047865 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047800    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:52.047779714 +0000 UTC m=+3.995130482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:51.048081 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047892    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:52.047878756 +0000 UTC m=+3.995229521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:51.048081 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.047911    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:52.047904025 +0000 UTC m=+3.995254786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:51.309528 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.309373    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c95b09e_3b2c_4064_a2b7_b55bcf4db6a4.slice/crio-c482447f98849f39b1a3982fa6b48a5587e7f88f9a322a9227584304236e424a WatchSource:0}: Error finding container c482447f98849f39b1a3982fa6b48a5587e7f88f9a322a9227584304236e424a: Status 404 returned error can't find the container with id c482447f98849f39b1a3982fa6b48a5587e7f88f9a322a9227584304236e424a
Sep 07 11:22:51.310689 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.310658    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2760e4f_b854_488a_9886_e92ade779208.slice/crio-6efde1eca380e38375ad0352cdc210a78db05f0248bf105367d0a913dfaf3ee2 WatchSource:0}: Error finding container 6efde1eca380e38375ad0352cdc210a78db05f0248bf105367d0a913dfaf3ee2: Status 404 returned error can't find the container with id 6efde1eca380e38375ad0352cdc210a78db05f0248bf105367d0a913dfaf3ee2
Sep 07 11:22:51.311732 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.311709    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3cbf66_5aad_433f_9f7a_79dc9c747517.slice/crio-7bb20c83e942e24523aae03d2a50678a6c79d0f1c52d9a9f0318f6dc7a7d03fd WatchSource:0}: Error finding container 7bb20c83e942e24523aae03d2a50678a6c79d0f1c52d9a9f0318f6dc7a7d03fd: Status 404 returned error can't find the container with id 7bb20c83e942e24523aae03d2a50678a6c79d0f1c52d9a9f0318f6dc7a7d03fd
Sep 07 11:22:51.313161 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.313132    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8d8134_6363_4b59_81c9_9b01c1b2df54.slice/crio-62689a15b4f0033def3f80f6485ba8492cb86af165daf9433aa6e5a1410c2f78 WatchSource:0}: Error finding container 62689a15b4f0033def3f80f6485ba8492cb86af165daf9433aa6e5a1410c2f78: Status 404 returned error can't find the container with id 62689a15b4f0033def3f80f6485ba8492cb86af165daf9433aa6e5a1410c2f78
Sep 07 11:22:51.316050 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.316031    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b017d57_3f8c_44a8_b277_b723bb14b44a.slice/crio-9f009a1725bedc0cbd2ca613418d69818a1742b22915e44b143dbb0031da6a38 WatchSource:0}: Error finding container 9f009a1725bedc0cbd2ca613418d69818a1742b22915e44b143dbb0031da6a38: Status 404 returned error can't find the container with id 9f009a1725bedc0cbd2ca613418d69818a1742b22915e44b143dbb0031da6a38
Sep 07 11:22:51.317354 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.317311    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda05961a1_9a71_424f_b728_7764722ae488.slice/crio-56bb727b042c8553d36c7e866b86a6252b566ab5c0decfb86564537111c0f5a7 WatchSource:0}: Error finding container 56bb727b042c8553d36c7e866b86a6252b566ab5c0decfb86564537111c0f5a7: Status 404 returned error can't find the container with id 56bb727b042c8553d36c7e866b86a6252b566ab5c0decfb86564537111c0f5a7
Sep 07 11:22:51.318269 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.318246    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c0a647_41c3_4e09_8060_ad1ae7940794.slice/crio-fe303b6640760ca216f67303cb4541248d63693f4b340d35d652353728db2545 WatchSource:0}: Error finding container fe303b6640760ca216f67303cb4541248d63693f4b340d35d652353728db2545: Status 404 returned error can't find the container with id fe303b6640760ca216f67303cb4541248d63693f4b340d35d652353728db2545
Sep 07 11:22:51.319855 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.319605    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796c1991_a31c_45ad_b16c_e3d0eeef05e1.slice/crio-a014ecd0ffd9544abf56a52c1871c7138c21ab1d7ab3703c557576716815a6dc WatchSource:0}: Error finding container a014ecd0ffd9544abf56a52c1871c7138c21ab1d7ab3703c557576716815a6dc: Status 404 returned error can't find the container with id a014ecd0ffd9544abf56a52c1871c7138c21ab1d7ab3703c557576716815a6dc
Sep 07 11:22:51.320888 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:22:51.320818    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5ad469_1623_4329_a48d_e71071315cdd.slice/crio-4302f541a47e3ebcb418bcd6c1dab0813b9d1b4b9a603cc98ea87bbd85d7bb22 WatchSource:0}: Error finding container 4302f541a47e3ebcb418bcd6c1dab0813b9d1b4b9a603cc98ea87bbd85d7bb22: Status 404 returned error can't find the container with id 4302f541a47e3ebcb418bcd6c1dab0813b9d1b4b9a603cc98ea87bbd85d7bb22
Sep 07 11:22:51.468998 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.468934    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 11:17:50 +0000 UTC" deadline="2028-02-13 18:57:22.14810826 +0000 UTC"
Sep 07 11:22:51.468998 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.468993    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12583h34m30.679120238s"
Sep 07 11:22:51.491677 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.491650    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:51.491840 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:51.491787    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:22:51.501869 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.501838    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lwmzh" event={"ID":"8d3cbf66-5aad-433f-9f7a-79dc9c747517","Type":"ContainerStarted","Data":"7bb20c83e942e24523aae03d2a50678a6c79d0f1c52d9a9f0318f6dc7a7d03fd"}
Sep 07 11:22:51.504526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.504332    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal" event={"ID":"422a2dc891c89e8aeb801cae87a1e08b","Type":"ContainerStarted","Data":"764292306e5d751ce7bd9d8c4df046b0c18585f909e4178c36fa001cd2ec9ce8"}
Sep 07 11:22:51.505557 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.505535    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts" event={"ID":"796c1991-a31c-45ad-b16c-e3d0eeef05e1","Type":"ContainerStarted","Data":"a014ecd0ffd9544abf56a52c1871c7138c21ab1d7ab3703c557576716815a6dc"}
Sep 07 11:22:51.506901 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.506879    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fjx89" event={"ID":"d0c0a647-41c3-4e09-8060-ad1ae7940794","Type":"ContainerStarted","Data":"fe303b6640760ca216f67303cb4541248d63693f4b340d35d652353728db2545"}
Sep 07 11:22:51.508145 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.508123    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"56bb727b042c8553d36c7e866b86a6252b566ab5c0decfb86564537111c0f5a7"}
Sep 07 11:22:51.509553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.509532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q4gvm" event={"ID":"4f8d8134-6363-4b59-81c9-9b01c1b2df54","Type":"ContainerStarted","Data":"62689a15b4f0033def3f80f6485ba8492cb86af165daf9433aa6e5a1410c2f78"}
Sep 07 11:22:51.510919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.510890    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerStarted","Data":"6efde1eca380e38375ad0352cdc210a78db05f0248bf105367d0a913dfaf3ee2"}
Sep 07 11:22:51.512293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.512274    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mvf88" event={"ID":"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4","Type":"ContainerStarted","Data":"c482447f98849f39b1a3982fa6b48a5587e7f88f9a322a9227584304236e424a"}
Sep 07 11:22:51.513478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.513459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lhhhk" event={"ID":"4f5ad469-1623-4329-a48d-e71071315cdd","Type":"ContainerStarted","Data":"4302f541a47e3ebcb418bcd6c1dab0813b9d1b4b9a603cc98ea87bbd85d7bb22"}
Sep 07 11:22:51.514544 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.514524    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7hfb" event={"ID":"2b017d57-3f8c-44a8-b277-b723bb14b44a","Type":"ContainerStarted","Data":"9f009a1725bedc0cbd2ca613418d69818a1742b22915e44b143dbb0031da6a38"}
Sep 07 11:22:51.526713 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:51.526677    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-45.ec2.internal" podStartSLOduration=2.526665516 podStartE2EDuration="2.526665516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:22:51.526549126 +0000 UTC m=+3.473899909" watchObservedRunningTime="2026-09-07 11:22:51.526665516 +0000 UTC m=+3.474016300"
Sep 07 11:22:52.055136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.055056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:52.055136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.055104    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:52.055351 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.055157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:52.055351 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055281    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:52.055351 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055348    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:54.055328635 +0000 UTC m=+6.002679410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:52.056140 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055766    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:52.056140 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055821    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:54.055804333 +0000 UTC m=+6.003155117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:52.056140 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055906    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:22:52.056140 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055920    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:22:52.056140 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055932    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:52.056140 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.055968    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:54.055956844 +0000 UTC m=+6.003307620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:52.393170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.393086    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:22:52.470901 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.470863    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 11:17:50 +0000 UTC" deadline="2028-04-20 18:48:04.720705926 +0000 UTC"
Sep 07 11:22:52.470901 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.470898    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14191h25m12.249811664s"
Sep 07 11:22:52.493725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.493699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:52.493907 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.493849    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:22:52.494229 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.494211    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:52.494359 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:52.494343    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:22:52.535710 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.535676    2560 generic.go:356] "Generic (PLEG): container finished" podID="00b2bc161ab733dadf328ac57d8c9e64" containerID="54b611e9ca6a0e9fc6c0db77a26e55d25d3d8d2e35c2b046d9fb9d535c82332e" exitCode=0
Sep 07 11:22:52.536786 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:52.536758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal" event={"ID":"00b2bc161ab733dadf328ac57d8c9e64","Type":"ContainerDied","Data":"54b611e9ca6a0e9fc6c0db77a26e55d25d3d8d2e35c2b046d9fb9d535c82332e"}
Sep 07 11:22:53.491856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:53.491774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:53.492508 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:53.491921    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:22:53.551704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:53.550977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal" event={"ID":"00b2bc161ab733dadf328ac57d8c9e64","Type":"ContainerStarted","Data":"8a0e99597fa21e4c7ba34429072290eaf6da0c028653db5d6dfb9637d81cc1ff"}
Sep 07 11:22:54.073902 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:54.073861    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:54.074061 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:54.073934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:54.074061 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:54.073965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:54.074151 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074131    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:22:54.074204 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074156    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:22:54.074204 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074170    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:54.074303 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074229    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:58.074210711 +0000 UTC m=+10.021561476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:54.074625 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074603    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:54.074708 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074656    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:58.074642506 +0000 UTC m=+10.021993271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:54.074870 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074711    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:54.074870 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.074760    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:22:58.074732707 +0000 UTC m=+10.022083470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:54.494401 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:54.493608    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:54.494401 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.493759    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:22:54.494401 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:54.493854    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:54.494401 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:54.493968    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:22:55.491912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:55.491570    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:55.491912 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:55.491709    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:22:56.492141 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:56.492098    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:56.492598 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:56.492246    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:22:56.492888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:56.492680    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:56.492888 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:56.492847    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:22:57.491877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:57.491771    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:57.492018 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:57.491876    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:22:58.112930 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:58.112734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:58.112930 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:58.112798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:58.112930 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:58.112849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:58.112930 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.112920    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.112968    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.112999    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:06.112979 +0000 UTC m=+18.060329779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.113004    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.113023    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:06.11301014 +0000 UTC m=+18.060360907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.113041    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.113055    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:58.113472 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.113141    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:06.113124385 +0000 UTC m=+18.060475167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:22:58.492637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:58.492560    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:22:58.493064 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.493021    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:22:58.493258 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:58.492903    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:22:58.493364 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:58.493338    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:22:59.491136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:22:59.491055    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:22:59.491589 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:22:59.491189    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:00.491797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:00.491751    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:00.492270 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:00.491887    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:00.492390 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:00.492355    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:00.492573 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:00.492503    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:01.491248 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:01.491217    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:01.491392 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:01.491328    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:02.491769 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:02.491723    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:02.492168 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:02.491728    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:02.492168 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:02.491849    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:02.492168 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:02.491975    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:03.491674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:03.491433    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:03.491858 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:03.491787    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:04.491871 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:04.491832    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:04.492322 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:04.491952    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:04.492322 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:04.491991    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:04.492322 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:04.492075    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:05.491477 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:05.491439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:05.491788 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:05.491567    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:06.171929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:06.171892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:06.171957    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:06.171979    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172071    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172093    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172111    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172126    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172138    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172149    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:22.172128162 +0000 UTC m=+34.119478938 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172171    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:22.172160105 +0000 UTC m=+34.119510871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:23:06.172341 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.172187    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:22.172177457 +0000 UTC m=+34.119528225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:23:06.491485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:06.491401    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:06.491658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:06.491400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:06.491658 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.491541    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:06.491658 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:06.491614    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:07.491832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:07.491794    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:07.492298 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:07.491904    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:08.491913 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:08.491880    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:08.491913 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:08.491904    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:08.492411 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:08.492004    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:08.492411 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:08.492119    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:09.491266 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.491151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:09.491361 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:09.491341    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:09.582834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.582812    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2760e4f-b854-488a-9886-e92ade779208" containerID="78bd83452fe7407713ba3e6c819af3afa382a2ca8eaee5d7d258cb9b015e1e73" exitCode=0
Sep 07 11:23:09.583422 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.582907    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerDied","Data":"78bd83452fe7407713ba3e6c819af3afa382a2ca8eaee5d7d258cb9b015e1e73"}
Sep 07 11:23:09.588039 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.588011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7hfb" event={"ID":"2b017d57-3f8c-44a8-b277-b723bb14b44a","Type":"ContainerStarted","Data":"c1f3d81ed1610307abbc0da75a6d77e140f39511ac379a518e2a28eaa6fbee46"}
Sep 07 11:23:09.589920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.589888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lwmzh" event={"ID":"8d3cbf66-5aad-433f-9f7a-79dc9c747517","Type":"ContainerStarted","Data":"f41abfda54ebc8a2567d1cca7bcece50f963a810fe9b99777b125c3bddc530ed"}
Sep 07 11:23:09.591587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.591565    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts" event={"ID":"796c1991-a31c-45ad-b16c-e3d0eeef05e1","Type":"ContainerStarted","Data":"9c8948c37469119c4ef4badca6ace1cb2dd2ae5a6e34d6fa26a7c6fa300e449e"}
Sep 07 11:23:09.593576 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.593561    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fjx89" event={"ID":"d0c0a647-41c3-4e09-8060-ad1ae7940794","Type":"ContainerStarted","Data":"e90ad0336b1d0227731a3a38af3013346b769f5e2c3cfe0fb85a6958bab0d477"}
Sep 07 11:23:09.596030 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.596011    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:23:09.596532 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.596440    2560 generic.go:356] "Generic (PLEG): container finished" podID="a05961a1-9a71-424f-b728-7764722ae488" containerID="930194da431b83912e6316eb2ac7dfdbeed967fc5f93088374329e6b6aee4dac" exitCode=1
Sep 07 11:23:09.596532 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.596504    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"e9fc0c34d897f2eb84c331812e776de40b6fb03f9b137bb17a2681c627837fa3"}
Sep 07 11:23:09.596677 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.596581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerDied","Data":"930194da431b83912e6316eb2ac7dfdbeed967fc5f93088374329e6b6aee4dac"}
Sep 07 11:23:09.596677 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.596633    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"7a9a7b370974f387c14f2748de470ac53ea5641654f436d5665d80bb1e0ecacb"}
Sep 07 11:23:09.598370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.598351    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q4gvm" event={"ID":"4f8d8134-6363-4b59-81c9-9b01c1b2df54","Type":"ContainerStarted","Data":"d6e61152853eb5d96d761f54fdc6cffb4c0a2f8f7b57d6d6ae7d0ab7bfdb6304"}
Sep 07 11:23:09.605149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.605108    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-45.ec2.internal" podStartSLOduration=20.605096928000002 podStartE2EDuration="20.605096928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:22:53.583195612 +0000 UTC m=+5.530546398" watchObservedRunningTime="2026-09-07 11:23:09.605096928 +0000 UTC m=+21.552447711"
Sep 07 11:23:09.623691 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.623640    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fjx89" podStartSLOduration=3.897374252 podStartE2EDuration="21.623627556s" totalImagesPullingTime="17.726253304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:09.623441418 +0000 UTC m=+21.570792201" watchObservedRunningTime="2026-09-07 11:23:09.623627556 +0000 UTC m=+21.570978339"
Sep 07 11:23:09.639938 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.639887    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m7hfb" podStartSLOduration=3.912111437 podStartE2EDuration="21.639845973s" totalImagesPullingTime="17.727734536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:09.639525751 +0000 UTC m=+21.586876543" watchObservedRunningTime="2026-09-07 11:23:09.639845973 +0000 UTC m=+21.587196758"
Sep 07 11:23:09.653674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.653621    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lwmzh" podStartSLOduration=11.378759401 podStartE2EDuration="21.653589418s" totalImagesPullingTime="10.274830017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:09.65314259 +0000 UTC m=+21.600493375" watchObservedRunningTime="2026-09-07 11:23:09.653589418 +0000 UTC m=+21.600940203"
Sep 07 11:23:09.667409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.667361    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q4gvm" podStartSLOduration=3.9515466139999997 podStartE2EDuration="21.667343037s" totalImagesPullingTime="17.715796423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:09.66678935 +0000 UTC m=+21.614140137" watchObservedRunningTime="2026-09-07 11:23:09.667343037 +0000 UTC m=+21.614693824"
Sep 07 11:23:09.753328 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.753143    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwmzh_8d3cbf66-5aad-433f-9f7a-79dc9c747517/konnectivity-agent/0.log" limit=1024
Sep 07 11:23:09.837692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:09.837657    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwmzh_8d3cbf66-5aad-433f-9f7a-79dc9c747517/konnectivity-agent/0.log" limit=1024
Sep 07 11:23:10.491652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.491369    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:10.491652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.491397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:10.491652 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:10.491496    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:10.491652 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:10.491607    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:10.518930 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.518899    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 11:23:10.601855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.601818    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mvf88" event={"ID":"2c95b09e-3b2c-4064-a2b7-b55bcf4db6a4","Type":"ContainerStarted","Data":"8b02ce5af54d3cc0ff5ffeaeb16712b3a8d8b5e6a9998c49d7c88c4e62abc997"}
Sep 07 11:23:10.603351 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.603329    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lhhhk" event={"ID":"4f5ad469-1623-4329-a48d-e71071315cdd","Type":"ContainerStarted","Data":"1352667dd5f2b552f217f97953c2b3e410453f4ebc102c84b4b698caf4cd62e0"}
Sep 07 11:23:10.605182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.605155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts" event={"ID":"796c1991-a31c-45ad-b16c-e3d0eeef05e1","Type":"ContainerStarted","Data":"aaaf1c3c043230bc67f7d645f55ab9d5f54b15713855e776fd89348652804285"}
Sep 07 11:23:10.607816 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.607801    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:23:10.608160 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.608133    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"94540db18180074c8f258fea9e2922429798b7e4a2e55dc5f3c9160ef4b6f1cd"}
Sep 07 11:23:10.608283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.608207    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"fa683cba015c7865ffe1326f78bb92828fb6a522e2a373695b06654ed1802c15"}
Sep 07 11:23:10.608283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.608259    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"7571ceb8e2debf1e51561091f2cd77a7194c227aa8824a35af059c9fe7ec81d3"}
Sep 07 11:23:10.617789 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.617729    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mvf88" podStartSLOduration=4.885485566 podStartE2EDuration="22.617711564s" totalImagesPullingTime="17.732225998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:10.617617828 +0000 UTC m=+22.564968613" watchObservedRunningTime="2026-09-07 11:23:10.617711564 +0000 UTC m=+22.565062348"
Sep 07 11:23:10.633875 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:10.633797    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lhhhk" podStartSLOduration=4.429908643 podStartE2EDuration="22.633779266s" totalImagesPullingTime="18.203870623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:10.63314712 +0000 UTC m=+22.580497903" watchObservedRunningTime="2026-09-07 11:23:10.633779266 +0000 UTC m=+22.581130050"
Sep 07 11:23:11.488216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:11.488094    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T11:23:10.518921679Z","UUID":"d94438ef-8149-46e3-93ab-7561f7b34220","Handler":null,"Name":"","Endpoint":""}
Sep 07 11:23:11.489833 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:11.489809    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 11:23:11.489961 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:11.489841    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 11:23:11.490904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:11.490881    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:11.491049 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:11.491031    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:12.490949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:12.490917    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:12.491475 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:12.491059    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:12.491475 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:12.491100    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:12.491475 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:12.491197    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:12.618102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:12.618064    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts" event={"ID":"796c1991-a31c-45ad-b16c-e3d0eeef05e1","Type":"ContainerStarted","Data":"512987744afff4b96d6d8104d3ddf4736e963952aa70f925d2484c04bf336426"}
Sep 07 11:23:12.622293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:12.622262    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:23:12.622797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:12.622775    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"25afc420ada864c2cfb3fecd4ec6496bfee3805bdcceb4c1abd8868e689cd620"}
Sep 07 11:23:12.635879 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:12.635836    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z26ts" podStartSLOduration=4.706899793 podStartE2EDuration="24.635819134s" totalImagesPullingTime="19.928919341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:12.635655274 +0000 UTC m=+24.583006058" watchObservedRunningTime="2026-09-07 11:23:12.635819134 +0000 UTC m=+24.583169919"
Sep 07 11:23:13.056077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:13.056041    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:23:13.056698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:13.056676    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:23:13.491584 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:13.491512    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:13.492047 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:13.491640    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:14.491447 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.491272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:14.491571 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.491275    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:14.491571 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:14.491531    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:14.491918 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:14.491661    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:14.494957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.494928    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:23:14.495078 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.494982    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lwmzh"
Sep 07 11:23:14.631449 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.631421    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:23:14.631904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.631874    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"5587fe4751f5e3a8149a87723d50f86692b50c412bce155ac2641cd17ef7cb5b"}
Sep 07 11:23:14.632214 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.632186    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:23:14.632370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.632353    2560 scope.go:117] "RemoveContainer" containerID="930194da431b83912e6316eb2ac7dfdbeed967fc5f93088374329e6b6aee4dac"
Sep 07 11:23:14.634224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.634192    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2760e4f-b854-488a-9886-e92ade779208" containerID="5465c5e799c975643bc196c4c60fd58e7ac328ef6859237926bc060294047cec" exitCode=0
Sep 07 11:23:14.634335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.634279    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerDied","Data":"5465c5e799c975643bc196c4c60fd58e7ac328ef6859237926bc060294047cec"}
Sep 07 11:23:14.648874 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:14.648853    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:23:15.491176 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.491143    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:15.491344 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:15.491245    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:15.641243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.641198    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:23:15.641866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.641636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" event={"ID":"a05961a1-9a71-424f-b728-7764722ae488","Type":"ContainerStarted","Data":"87b2ff193cbff83db5007f7a169b7d6625b770b8d05bf313f72aa01e5cd960af"}
Sep 07 11:23:15.641866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.641850    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:23:15.641969 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.641927    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:23:15.657979 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.657953    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:23:15.671498 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:15.671456    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx" podStartSLOduration=9.926192341 podStartE2EDuration="27.671441489s" totalImagesPullingTime="17.745249148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:15.671328728 +0000 UTC m=+27.618679512" watchObservedRunningTime="2026-09-07 11:23:15.671441489 +0000 UTC m=+27.618792276"
Sep 07 11:23:16.046693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.046640    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6dsdg"]
Sep 07 11:23:16.046964 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.046798    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:16.046964 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:16.046918    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:16.050133 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.050074    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6628l"]
Sep 07 11:23:16.050244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.050184    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:16.050294 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:16.050272    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:16.050852 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.050824    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bjfn8"]
Sep 07 11:23:16.050955 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.050930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:16.051054 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:16.051031    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:16.649356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.649138    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2760e4f-b854-488a-9886-e92ade779208" containerID="3703a7600845189479546f732a456b94d558b52716f32229752937fe2624006e" exitCode=0
Sep 07 11:23:16.649356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:16.649218    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerDied","Data":"3703a7600845189479546f732a456b94d558b52716f32229752937fe2624006e"}
Sep 07 11:23:17.491319 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:17.491287    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:17.491471 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:17.491288    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:17.491521 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:17.491480    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:17.491521 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:17.491390    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:17.491631 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:17.491532    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:17.491631 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:17.491611    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:18.655716 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:18.655678    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2760e4f-b854-488a-9886-e92ade779208" containerID="ee520afbfe88dd01a95456045ccec2725bcb3b51cb0c860026d30475f6d1b301" exitCode=0
Sep 07 11:23:18.656069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:18.655734    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerDied","Data":"ee520afbfe88dd01a95456045ccec2725bcb3b51cb0c860026d30475f6d1b301"}
Sep 07 11:23:19.491506 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:19.491471    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:19.491710 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:19.491471    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:19.491710 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:19.491587    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:19.491710 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:19.491617    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:19.491710 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:19.491673    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:19.491968 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:19.491792    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:21.491127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:21.491098    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:21.491127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:21.491130    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:21.491535 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:21.491186    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6628l" podUID="bbeb8bc7-bc7f-47ac-92e6-040b1c287c21"
Sep 07 11:23:21.491535 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:21.491221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:21.491535 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:21.491293    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bjfn8" podUID="9e39c68a-5691-47a5-bb3c-fa289e615054"
Sep 07 11:23:21.491535 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:21.491352    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6dsdg" podUID="62b756ad-9c5d-493f-9afa-553c4a741503"
Sep 07 11:23:22.189239 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.189148    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:22.189239 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.189210    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.189244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189295    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189361    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret podName:62b756ad-9c5d-493f-9afa-553c4a741503 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:54.189341785 +0000 UTC m=+66.136692553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret") pod "global-pull-secret-syncer-6dsdg" (UID: "62b756ad-9c5d-493f-9afa-553c4a741503") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189362    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189368    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189380    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189394    2560 projected.go:196] Error preparing data for projected volume kube-api-access-cs6b6 for pod openshift-network-diagnostics/network-check-target-6628l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189433    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:54.189414766 +0000 UTC m=+66.136765553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:23:22.189457 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.189452    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6 podName:bbeb8bc7-bc7f-47ac-92e6-040b1c287c21 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:54.189442167 +0000 UTC m=+66.136792935 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cs6b6" (UniqueName: "kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6") pod "network-check-target-6628l" (UID: "bbeb8bc7-bc7f-47ac-92e6-040b1c287c21") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:23:22.382936 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.382897    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-45.ec2.internal" event="NodeReady"
Sep 07 11:23:22.383168 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.383064    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 11:23:22.425160 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.425125    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q7r6q"]
Sep 07 11:23:22.448798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.448695    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sdzvg"]
Sep 07 11:23:22.448948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.448813    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.451042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.451022    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ph89j\""
Sep 07 11:23:22.451175 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.451020    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 11:23:22.451236 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.451198    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 11:23:22.462551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.462532    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q7r6q"]
Sep 07 11:23:22.462652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.462562    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sdzvg"]
Sep 07 11:23:22.462652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.462629    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.465065 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.465042    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 11:23:22.465206 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.465184    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 11:23:22.465307 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.465288    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4j7np\""
Sep 07 11:23:22.465426 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.465338    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 11:23:22.592640 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.592599    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30549a7f-5111-45f6-ba9f-13729a5d9322-cert\") pod \"ingress-canary-sdzvg\" (UID: \"30549a7f-5111-45f6-ba9f-13729a5d9322\") " pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.592640 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.592644    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84e6938a-fe71-410a-aa30-9a929b1be9a9-config-volume\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.593115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.592702    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzcnt\" (UniqueName: \"kubernetes.io/projected/84e6938a-fe71-410a-aa30-9a929b1be9a9-kube-api-access-mzcnt\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.593115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.592755    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmmj\" (UniqueName: \"kubernetes.io/projected/30549a7f-5111-45f6-ba9f-13729a5d9322-kube-api-access-wnmmj\") pod \"ingress-canary-sdzvg\" (UID: \"30549a7f-5111-45f6-ba9f-13729a5d9322\") " pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.593115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.592780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.593115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.592803    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e6938a-fe71-410a-aa30-9a929b1be9a9-tmp-dir\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.693545 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693507    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmmj\" (UniqueName: \"kubernetes.io/projected/30549a7f-5111-45f6-ba9f-13729a5d9322-kube-api-access-wnmmj\") pod \"ingress-canary-sdzvg\" (UID: \"30549a7f-5111-45f6-ba9f-13729a5d9322\") " pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.693763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.693763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e6938a-fe71-410a-aa30-9a929b1be9a9-tmp-dir\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.693763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693646    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30549a7f-5111-45f6-ba9f-13729a5d9322-cert\") pod \"ingress-canary-sdzvg\" (UID: \"30549a7f-5111-45f6-ba9f-13729a5d9322\") " pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.693763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84e6938a-fe71-410a-aa30-9a929b1be9a9-config-volume\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.693763 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.693684    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:22.693763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzcnt\" (UniqueName: \"kubernetes.io/projected/84e6938a-fe71-410a-aa30-9a929b1be9a9-kube-api-access-mzcnt\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.694062 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:22.693768    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:23.193732927 +0000 UTC m=+35.141083701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:22.694062 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.693953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e6938a-fe71-410a-aa30-9a929b1be9a9-tmp-dir\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.694358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.694314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84e6938a-fe71-410a-aa30-9a929b1be9a9-config-volume\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.699028 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.698807    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30549a7f-5111-45f6-ba9f-13729a5d9322-cert\") pod \"ingress-canary-sdzvg\" (UID: \"30549a7f-5111-45f6-ba9f-13729a5d9322\") " pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.704531 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.704511    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzcnt\" (UniqueName: \"kubernetes.io/projected/84e6938a-fe71-410a-aa30-9a929b1be9a9-kube-api-access-mzcnt\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:22.704647 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.704536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmmj\" (UniqueName: \"kubernetes.io/projected/30549a7f-5111-45f6-ba9f-13729a5d9322-kube-api-access-wnmmj\") pod \"ingress-canary-sdzvg\" (UID: \"30549a7f-5111-45f6-ba9f-13729a5d9322\") " pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:22.772509 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:22.772476    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sdzvg"
Sep 07 11:23:23.199312 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.199285    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:23.199442 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:23.199418    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:23.199483 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:23.199474    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:24.199459734 +0000 UTC m=+36.146810507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:23.390899 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.390865    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sdzvg"]
Sep 07 11:23:23.395446 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:23:23.395424    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30549a7f_5111_45f6_ba9f_13729a5d9322.slice/crio-70b28dd27143a7eed1e05b3183ee946e920996d529e2ad6f12fd15458e074a78 WatchSource:0}: Error finding container 70b28dd27143a7eed1e05b3183ee946e920996d529e2ad6f12fd15458e074a78: Status 404 returned error can't find the container with id 70b28dd27143a7eed1e05b3183ee946e920996d529e2ad6f12fd15458e074a78
Sep 07 11:23:23.491056 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.491022    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:23.491056 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.491040    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:23.491209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.491194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:23.494024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.494008    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 11:23:23.494109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.494010    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 11:23:23.494641 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.494617    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q8r9c\""
Sep 07 11:23:23.494752 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.494661    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 11:23:23.494938 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.494924    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 11:23:23.495004 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.494928    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qlhnv\""
Sep 07 11:23:23.670582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.670505    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sdzvg" event={"ID":"30549a7f-5111-45f6-ba9f-13729a5d9322","Type":"ContainerStarted","Data":"70b28dd27143a7eed1e05b3183ee946e920996d529e2ad6f12fd15458e074a78"}
Sep 07 11:23:23.673302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.673278    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2760e4f-b854-488a-9886-e92ade779208" containerID="0c784c42522ee49c72a12f59e4778ef55cc0f0443c032b6cd226321b48f983f9" exitCode=0
Sep 07 11:23:23.673397 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:23.673310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerDied","Data":"0c784c42522ee49c72a12f59e4778ef55cc0f0443c032b6cd226321b48f983f9"}
Sep 07 11:23:24.208864 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:24.208832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:24.209029 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:24.208959    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:24.209029 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:24.209016    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:26.209000663 +0000 UTC m=+38.156351424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:24.680025 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:24.679988    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2760e4f-b854-488a-9886-e92ade779208" containerID="87deeb99074a061a65ddcbb898bfe40c38ecca04870c47816e9898ccb8b49811" exitCode=0
Sep 07 11:23:24.680484 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:24.680046    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerDied","Data":"87deeb99074a061a65ddcbb898bfe40c38ecca04870c47816e9898ccb8b49811"}
Sep 07 11:23:25.686292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:25.686135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrp82" event={"ID":"e2760e4f-b854-488a-9886-e92ade779208","Type":"ContainerStarted","Data":"5b629ccdf82a9a923a4cc654d8e76c8a6aabfd07e9ecad61f68942de1d845ec7"}
Sep 07 11:23:25.709819 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:25.709716    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zrp82" podStartSLOduration=8.181366941 podStartE2EDuration="37.709696128s" totalImagesPullingTime="29.528329187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:25.708391345 +0000 UTC m=+37.655742128" watchObservedRunningTime="2026-09-07 11:23:25.709696128 +0000 UTC m=+37.657046908"
Sep 07 11:23:26.224670 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:26.224637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:26.224844 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:26.224810    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:26.224885 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:26.224869    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:30.224854958 +0000 UTC m=+42.172205724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:26.690480 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:26.690444    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sdzvg" event={"ID":"30549a7f-5111-45f6-ba9f-13729a5d9322","Type":"ContainerStarted","Data":"cfece33d2c815f6fd3f6946a4dff2f47d7c45e0d9af8397673046e03129025cb"}
Sep 07 11:23:30.253680 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:30.253644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:30.254056 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:30.253832    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:30.254056 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:30.253898    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:38.253882651 +0000 UTC m=+50.201233413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:38.305587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:38.305551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:38.306010 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:38.305705    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:38.306010 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:38.305787    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:23:54.305770141 +0000 UTC m=+66.253120904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:47.668359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:47.668328    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8clx"
Sep 07 11:23:47.695733 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:47.695687    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sdzvg" podStartSLOduration=23.475746862 podStartE2EDuration="25.695672567s" totalImagesPullingTime="2.219925705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:26.705342345 +0000 UTC m=+38.652693128" watchObservedRunningTime="2026-09-07 11:23:47.695672567 +0000 UTC m=+59.643023351"
Sep 07 11:23:54.212821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.212785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:23:54.212821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.212825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:54.213367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.212867    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:54.215695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.215673    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 11:23:54.215695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.215690    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 11:23:54.215822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.215787    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 11:23:54.223476 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:54.223451    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:23:54.223587 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:54.223505    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs podName:9e39c68a-5691-47a5-bb3c-fa289e615054 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:58.223489968 +0000 UTC m=+130.170840729 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs") pod "network-metrics-daemon-bjfn8" (UID: "9e39c68a-5691-47a5-bb3c-fa289e615054") : secret "metrics-daemon-secret" not found
Sep 07 11:23:54.225350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.225334    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 11:23:54.226610 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.226589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62b756ad-9c5d-493f-9afa-553c4a741503-original-pull-secret\") pod \"global-pull-secret-syncer-6dsdg\" (UID: \"62b756ad-9c5d-493f-9afa-553c4a741503\") " pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:54.235651 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.235628    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs6b6\" (UniqueName: \"kubernetes.io/projected/bbeb8bc7-bc7f-47ac-92e6-040b1c287c21-kube-api-access-cs6b6\") pod \"network-check-target-6628l\" (UID: \"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21\") " pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:54.313803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.313769    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:23:54.313932 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:54.313909    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:23:54.313973 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:23:54.313959    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls podName:84e6938a-fe71-410a-aa30-9a929b1be9a9 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:26.313946265 +0000 UTC m=+98.261297027 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls") pod "dns-default-q7r6q" (UID: "84e6938a-fe71-410a-aa30-9a929b1be9a9") : secret "dns-default-metrics-tls" not found
Sep 07 11:23:54.399977 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.399945    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6dsdg"
Sep 07 11:23:54.413399 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.413371    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q8r9c\""
Sep 07 11:23:54.422059 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.422037    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:54.687615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.687585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6dsdg"]
Sep 07 11:23:54.690314 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:23:54.690289    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b756ad_9c5d_493f_9afa_553c4a741503.slice/crio-fda955fe7c515b97765416bdff73923b728f4e016abd5039dbba2a9c93c58500 WatchSource:0}: Error finding container fda955fe7c515b97765416bdff73923b728f4e016abd5039dbba2a9c93c58500: Status 404 returned error can't find the container with id fda955fe7c515b97765416bdff73923b728f4e016abd5039dbba2a9c93c58500
Sep 07 11:23:54.713667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.713637    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6628l"]
Sep 07 11:23:54.717906 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:23:54.717858    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbeb8bc7_bc7f_47ac_92e6_040b1c287c21.slice/crio-5de7144c4c3a4e216121f8cebe44cd3038e1573263f851545dc6dd9d8b06f16f WatchSource:0}: Error finding container 5de7144c4c3a4e216121f8cebe44cd3038e1573263f851545dc6dd9d8b06f16f: Status 404 returned error can't find the container with id 5de7144c4c3a4e216121f8cebe44cd3038e1573263f851545dc6dd9d8b06f16f
Sep 07 11:23:54.755918 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.755877    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6628l" event={"ID":"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21","Type":"ContainerStarted","Data":"5de7144c4c3a4e216121f8cebe44cd3038e1573263f851545dc6dd9d8b06f16f"}
Sep 07 11:23:54.757043 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:54.757013    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6dsdg" event={"ID":"62b756ad-9c5d-493f-9afa-553c4a741503","Type":"ContainerStarted","Data":"fda955fe7c515b97765416bdff73923b728f4e016abd5039dbba2a9c93c58500"}
Sep 07 11:23:59.772934 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:59.772892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6628l" event={"ID":"bbeb8bc7-bc7f-47ac-92e6-040b1c287c21","Type":"ContainerStarted","Data":"066d794da4236bb1a3cacebcd08aa262ed7dec544efde120bf45116c432a01a8"}
Sep 07 11:23:59.773356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:59.773035    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:23:59.774362 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:59.774340    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6dsdg" event={"ID":"62b756ad-9c5d-493f-9afa-553c4a741503","Type":"ContainerStarted","Data":"63c3eabf9b7dface4283d335eb5c58ead892004f53b6e023b77a7c9b2a792c3a"}
Sep 07 11:23:59.788425 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:59.788382    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6628l" podStartSLOduration=67.092271805 podStartE2EDuration="1m11.788353366s" totalImagesPullingTime="4.696081561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:59.78829678 +0000 UTC m=+71.735647564" watchObservedRunningTime="2026-09-07 11:23:59.788353366 +0000 UTC m=+71.735704167"
Sep 07 11:23:59.803533 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:23:59.803491    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6dsdg" podStartSLOduration=67.076636026 podStartE2EDuration="1m11.803478432s" totalImagesPullingTime="4.726842406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:23:59.802758552 +0000 UTC m=+71.750109330" watchObservedRunningTime="2026-09-07 11:23:59.803478432 +0000 UTC m=+71.750829216"
Sep 07 11:24:01.989578 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:01.989547    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"]
Sep 07 11:24:01.991576 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:01.991560    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"
Sep 07 11:24:01.993878 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:01.993848    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:24:01.994447 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:01.994427    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 07 11:24:01.994539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:01.994428    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gp8zq\""
Sep 07 11:24:02.002429 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.002407    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"]
Sep 07 11:24:02.062171 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.062135    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvrnx\" (UniqueName: \"kubernetes.io/projected/60e08d47-09d8-49fd-a7a5-afe61305526d-kube-api-access-gvrnx\") pod \"volume-data-source-validator-695c895bd6-54rgr\" (UID: \"60e08d47-09d8-49fd-a7a5-afe61305526d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"
Sep 07 11:24:02.068792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.068763    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"]
Sep 07 11:24:02.071056 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.071036    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"]
Sep 07 11:24:02.071201 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.071183    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.072904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.072884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.073354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.073335    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 07 11:24:02.073622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.073597    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 07 11:24:02.073700 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.073599    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:24:02.074170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.074150    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wklr8\""
Sep 07 11:24:02.075466 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.075451    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 07 11:24:02.075566 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.075477    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 07 11:24:02.075727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.075707    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v974h\""
Sep 07 11:24:02.075970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.075955    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 07 11:24:02.081269 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.081250    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 07 11:24:02.088841 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.088820    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"]
Sep 07 11:24:02.092243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.092225    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"]
Sep 07 11:24:02.162885 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.162850    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l99pc\" (UniqueName: \"kubernetes.io/projected/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-kube-api-access-l99pc\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.162885 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.162886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-image-registry-private-configuration\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.162906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.162921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmnwd\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-kube-api-access-tmnwd\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.162985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvrnx\" (UniqueName: \"kubernetes.io/projected/60e08d47-09d8-49fd-a7a5-afe61305526d-kube-api-access-gvrnx\") pod \"volume-data-source-validator-695c895bd6-54rgr\" (UID: \"60e08d47-09d8-49fd-a7a5-afe61305526d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"
Sep 07 11:24:02.163127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.163068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-certificates\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.163107    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-bound-sa-token\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.163143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-ca-trust-extracted\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.163186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.163352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.163244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-installation-pull-secrets\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.163352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.163289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-trusted-ca\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.170608 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.170571    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvrnx\" (UniqueName: \"kubernetes.io/projected/60e08d47-09d8-49fd-a7a5-afe61305526d-kube-api-access-gvrnx\") pod \"volume-data-source-validator-695c895bd6-54rgr\" (UID: \"60e08d47-09d8-49fd-a7a5-afe61305526d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"
Sep 07 11:24:02.263888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.263888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-installation-pull-secrets\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.263888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-trusted-ca\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.263888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l99pc\" (UniqueName: \"kubernetes.io/projected/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-kube-api-access-l99pc\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-image-registry-private-configuration\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263952    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmnwd\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-kube-api-access-tmnwd\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.263972    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.264037    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.264055    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b7f6f5dd-5vs9k: secret "image-registry-tls" not found
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.264058    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls podName:3fa80310-9705-4c3c-9c57-a1e8b46ffca8 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:02.764037612 +0000 UTC m=+74.711388373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-n4g2g" (UID: "3fa80310-9705-4c3c-9c57-a1e8b46ffca8") : secret "samples-operator-tls" not found
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.264163    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls podName:e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:02.764145385 +0000 UTC m=+74.711496154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls") pod "image-registry-59b7f6f5dd-5vs9k" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55") : secret "image-registry-tls" not found
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.263982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-certificates\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.264209    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-bound-sa-token\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264683 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.264252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-ca-trust-extracted\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264683 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.264667    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-ca-trust-extracted\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.264768 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.264701    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-certificates\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.265127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.265105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-trusted-ca\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.266331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.266307    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-image-registry-private-configuration\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.266412 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.266335    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-installation-pull-secrets\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.272881 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.272858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmnwd\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-kube-api-access-tmnwd\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.273688 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.273662    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l99pc\" (UniqueName: \"kubernetes.io/projected/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-kube-api-access-l99pc\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.273906 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.273884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-bound-sa-token\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.299615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.299588    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"
Sep 07 11:24:02.564818 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.564723    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr"]
Sep 07 11:24:02.568771 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:02.568728    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e08d47_09d8_49fd_a7a5_afe61305526d.slice/crio-daad1bccb96632bd528f854d6eb10462d936a8ff3602a151a9c90e73f6fa8030 WatchSource:0}: Error finding container daad1bccb96632bd528f854d6eb10462d936a8ff3602a151a9c90e73f6fa8030: Status 404 returned error can't find the container with id daad1bccb96632bd528f854d6eb10462d936a8ff3602a151a9c90e73f6fa8030
Sep 07 11:24:02.767894 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.767855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:02.768095 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.767925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:02.768095 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.768022    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 11:24:02.768095 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.768037    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:24:02.768095 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.768059    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b7f6f5dd-5vs9k: secret "image-registry-tls" not found
Sep 07 11:24:02.768095 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.768075    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls podName:3fa80310-9705-4c3c-9c57-a1e8b46ffca8 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:03.768061578 +0000 UTC m=+75.715412340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-n4g2g" (UID: "3fa80310-9705-4c3c-9c57-a1e8b46ffca8") : secret "samples-operator-tls" not found
Sep 07 11:24:02.768283 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:02.768109    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls podName:e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:03.768091265 +0000 UTC m=+75.715442043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls") pod "image-registry-59b7f6f5dd-5vs9k" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55") : secret "image-registry-tls" not found
Sep 07 11:24:02.782016 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:02.781984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr" event={"ID":"60e08d47-09d8-49fd-a7a5-afe61305526d","Type":"ContainerStarted","Data":"daad1bccb96632bd528f854d6eb10462d936a8ff3602a151a9c90e73f6fa8030"}
Sep 07 11:24:03.662814 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.662779    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-8lnf8"]
Sep 07 11:24:03.665659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.665633    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:03.667879 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.667852    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 07 11:24:03.667879 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.667862    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 07 11:24:03.668033 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.667946    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-p5vbz\""
Sep 07 11:24:03.674293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.674096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-nginx-conf\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:03.674293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.674131    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:03.675858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.675835    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-8lnf8"]
Sep 07 11:24:03.774639 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.774616    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:03.774775 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.774692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:03.774775 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.774756    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-nginx-conf\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:03.774873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.774783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:03.774873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774815    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 11:24:03.774873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774832    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:24:03.774873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774852    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b7f6f5dd-5vs9k: secret "image-registry-tls" not found
Sep 07 11:24:03.774873 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774873    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls podName:3fa80310-9705-4c3c-9c57-a1e8b46ffca8 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:05.774856982 +0000 UTC m=+77.722207755 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-n4g2g" (UID: "3fa80310-9705-4c3c-9c57-a1e8b46ffca8") : secret "samples-operator-tls" not found
Sep 07 11:24:03.775041 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774894    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 11:24:03.775041 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774899    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls podName:e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:05.774881683 +0000 UTC m=+77.722232459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls") pod "image-registry-59b7f6f5dd-5vs9k" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55") : secret "image-registry-tls" not found
Sep 07 11:24:03.775041 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:03.774942    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert podName:151e67a0-7edf-4a1a-a0a0-d16fc7ded050 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:04.274933903 +0000 UTC m=+76.222284668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-8lnf8" (UID: "151e67a0-7edf-4a1a-a0a0-d16fc7ded050") : secret "networking-console-plugin-cert" not found
Sep 07 11:24:03.775540 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.775519    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-nginx-conf\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:03.785692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.785670    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr" event={"ID":"60e08d47-09d8-49fd-a7a5-afe61305526d","Type":"ContainerStarted","Data":"21cc58c0b79b2618434e695208d2f38988dde51d634f0d52b627d3d7aa7038bd"}
Sep 07 11:24:03.801847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:03.801793    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-54rgr" podStartSLOduration=1.6594102290000001 podStartE2EDuration="2.801776319s" totalImagesPullingTime="1.14236609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:03.801543553 +0000 UTC m=+75.748894338" watchObservedRunningTime="2026-09-07 11:24:03.801776319 +0000 UTC m=+75.749127104"
Sep 07 11:24:04.278368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:04.278317    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:04.278523 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:04.278463    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 11:24:04.278581 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:04.278530    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert podName:151e67a0-7edf-4a1a-a0a0-d16fc7ded050 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:05.278514638 +0000 UTC m=+77.225865404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-8lnf8" (UID: "151e67a0-7edf-4a1a-a0a0-d16fc7ded050") : secret "networking-console-plugin-cert" not found
Sep 07 11:24:05.285003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:05.284958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:05.285449 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.285077    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 11:24:05.285449 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.285131    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert podName:151e67a0-7edf-4a1a-a0a0-d16fc7ded050 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:07.285115758 +0000 UTC m=+79.232466527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-8lnf8" (UID: "151e67a0-7edf-4a1a-a0a0-d16fc7ded050") : secret "networking-console-plugin-cert" not found
Sep 07 11:24:05.788954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:05.788918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:05.789128 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:05.788993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:05.789128 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.789060    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:24:05.789128 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.789075    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b7f6f5dd-5vs9k: secret "image-registry-tls" not found
Sep 07 11:24:05.789128 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.789102    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 11:24:05.789128 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.789128    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls podName:e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:09.78911192 +0000 UTC m=+81.736462687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls") pod "image-registry-59b7f6f5dd-5vs9k" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55") : secret "image-registry-tls" not found
Sep 07 11:24:05.789387 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:05.789152    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls podName:3fa80310-9705-4c3c-9c57-a1e8b46ffca8 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:09.789135636 +0000 UTC m=+81.736486397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-n4g2g" (UID: "3fa80310-9705-4c3c-9c57-a1e8b46ffca8") : secret "samples-operator-tls" not found
Sep 07 11:24:07.299691 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.299658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:07.300179 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:07.299857    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 11:24:07.300179 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:07.299947    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert podName:151e67a0-7edf-4a1a-a0a0-d16fc7ded050 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:11.299917754 +0000 UTC m=+83.247268537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-8lnf8" (UID: "151e67a0-7edf-4a1a-a0a0-d16fc7ded050") : secret "networking-console-plugin-cert" not found
Sep 07 11:24:07.393819 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.393787    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4lks4"]
Sep 07 11:24:07.396063 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.396046    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.398314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.398286    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 11:24:07.398412 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.398330    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 11:24:07.398967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.398948    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6gln4\""
Sep 07 11:24:07.399024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.398969    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 11:24:07.399024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.398985    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 11:24:07.408282 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.408261    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4lks4"]
Sep 07 11:24:07.502149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.502111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-crio-socket\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.502294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.502165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-data-volume\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.502294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.502188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wdw\" (UniqueName: \"kubernetes.io/projected/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-api-access-b5wdw\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.502294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.502262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.502294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.502284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.602992 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.602918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-crio-socket\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.602992 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.602956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-data-volume\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.602992 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.602978    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wdw\" (UniqueName: \"kubernetes.io/projected/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-api-access-b5wdw\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.603197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.603038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-crio-socket\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.603197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.603046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.603197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.603105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.603197 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:07.603113    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:24:07.603197 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:07.603181    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:08.103164088 +0000 UTC m=+80.050514864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : configmap "kube-rbac-proxy" not found
Sep 07 11:24:07.603197 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:07.603186    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:07.603459 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:07.603231    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:08.103218415 +0000 UTC m=+80.050569177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:07.603459 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.603355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-data-volume\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:07.610389 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:07.610369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wdw\" (UniqueName: \"kubernetes.io/projected/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-api-access-b5wdw\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:08.025229 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.025199    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"]
Sep 07 11:24:08.027365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.027351    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"
Sep 07 11:24:08.029929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.029904    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 07 11:24:08.029929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.029904    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 07 11:24:08.030556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.030536    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zcqtm\""
Sep 07 11:24:08.040361 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.040342    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"]
Sep 07 11:24:08.107251 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.107218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:08.107251 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.107251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:08.107434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.107294    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpqvr\" (UniqueName: \"kubernetes.io/projected/b456dadd-b2de-4b84-99d8-69785d6ecafa-kube-api-access-xpqvr\") pod \"migrator-fdbb8fdff-knz5r\" (UID: \"b456dadd-b2de-4b84-99d8-69785d6ecafa\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"
Sep 07 11:24:08.107434 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:08.107342    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:24:08.107434 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:08.107381    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:08.107434 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:08.107407    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:09.107388834 +0000 UTC m=+81.054739827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : configmap "kube-rbac-proxy" not found
Sep 07 11:24:08.107566 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:08.107440    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:09.107427203 +0000 UTC m=+81.054777965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:08.208048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.208014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpqvr\" (UniqueName: \"kubernetes.io/projected/b456dadd-b2de-4b84-99d8-69785d6ecafa-kube-api-access-xpqvr\") pod \"migrator-fdbb8fdff-knz5r\" (UID: \"b456dadd-b2de-4b84-99d8-69785d6ecafa\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"
Sep 07 11:24:08.216128 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.216099    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpqvr\" (UniqueName: \"kubernetes.io/projected/b456dadd-b2de-4b84-99d8-69785d6ecafa-kube-api-access-xpqvr\") pod \"migrator-fdbb8fdff-knz5r\" (UID: \"b456dadd-b2de-4b84-99d8-69785d6ecafa\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"
Sep 07 11:24:08.336198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.336122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"
Sep 07 11:24:08.600618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.600536    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r"]
Sep 07 11:24:08.604073 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:08.604046    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb456dadd_b2de_4b84_99d8_69785d6ecafa.slice/crio-f2701526f4b24534ff045bda92640426778a2d4cab37a4b1d4e2b9367d19cc6b WatchSource:0}: Error finding container f2701526f4b24534ff045bda92640426778a2d4cab37a4b1d4e2b9367d19cc6b: Status 404 returned error can't find the container with id f2701526f4b24534ff045bda92640426778a2d4cab37a4b1d4e2b9367d19cc6b
Sep 07 11:24:08.799486 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:08.799453    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r" event={"ID":"b456dadd-b2de-4b84-99d8-69785d6ecafa","Type":"ContainerStarted","Data":"f2701526f4b24534ff045bda92640426778a2d4cab37a4b1d4e2b9367d19cc6b"}
Sep 07 11:24:09.113404 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:09.113365    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:09.113404 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:09.113409    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:09.113605 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.113498    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:24:09.113605 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.113535    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:09.113605 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.113568    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:11.113548018 +0000 UTC m=+83.060898800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : configmap "kube-rbac-proxy" not found
Sep 07 11:24:09.113605 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.113586    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:11.113576719 +0000 UTC m=+83.060927484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:09.132914 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:09.132890    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fjx89_d0c0a647-41c3-4e09-8060-ad1ae7940794/dns-node-resolver/0.log"
Sep 07 11:24:09.819889 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:09.819868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:09.820258 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:09.819964    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:09.820258 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.820011    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 11:24:09.820258 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.820067    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls podName:3fa80310-9705-4c3c-9c57-a1e8b46ffca8 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:17.820052044 +0000 UTC m=+89.767402813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-n4g2g" (UID: "3fa80310-9705-4c3c-9c57-a1e8b46ffca8") : secret "samples-operator-tls" not found
Sep 07 11:24:09.820258 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.820095    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:24:09.820258 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.820110    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b7f6f5dd-5vs9k: secret "image-registry-tls" not found
Sep 07 11:24:09.820258 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:09.820163    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls podName:e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:17.82014629 +0000 UTC m=+89.767497053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls") pod "image-registry-59b7f6f5dd-5vs9k" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55") : secret "image-registry-tls" not found
Sep 07 11:24:10.333177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:10.333149    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q4gvm_4f8d8134-6363-4b59-81c9-9b01c1b2df54/node-ca/0.log"
Sep 07 11:24:10.806659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:10.806628    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r" event={"ID":"b456dadd-b2de-4b84-99d8-69785d6ecafa","Type":"ContainerStarted","Data":"b4e43e5139e9fed6b72f963bb5f235c3d32732bf4c0700434284b728fb483daf"}
Sep 07 11:24:10.806839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:10.806677    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r" event={"ID":"b456dadd-b2de-4b84-99d8-69785d6ecafa","Type":"ContainerStarted","Data":"d667bca7b415aa36f9d61e59cf5fb9c29053f2d56e2f3ee8c0af28c580590492"}
Sep 07 11:24:10.820712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:10.820665    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-knz5r" podStartSLOduration=1.610373222 podStartE2EDuration="2.820649644s" totalImagesPullingTime="1.210276422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:10.820161153 +0000 UTC m=+82.767511939" watchObservedRunningTime="2026-09-07 11:24:10.820649644 +0000 UTC m=+82.768000428"
Sep 07 11:24:11.129611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:11.129513    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:11.129611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:11.129558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:11.129886 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:11.129649    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:11.129886 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:11.129652    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:24:11.129886 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:11.129704    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:15.129691222 +0000 UTC m=+87.077041985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:11.129886 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:11.129717    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:15.12971126 +0000 UTC m=+87.077062022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : configmap "kube-rbac-proxy" not found
Sep 07 11:24:11.330673 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:11.330638    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:11.330876 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:11.330797    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 11:24:11.330876 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:11.330863    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert podName:151e67a0-7edf-4a1a-a0a0-d16fc7ded050 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:19.33084698 +0000 UTC m=+91.278197747 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-8lnf8" (UID: "151e67a0-7edf-4a1a-a0a0-d16fc7ded050") : secret "networking-console-plugin-cert" not found
Sep 07 11:24:11.333091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:11.333055    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sdzvg_30549a7f-5111-45f6-ba9f-13729a5d9322/serve-healthcheck-canary/0.log"
Sep 07 11:24:15.161674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:15.161627    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:15.161674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:15.161694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:15.162169 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:15.161761    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:24:15.162169 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:15.161818    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:23.161804352 +0000 UTC m=+95.109155119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : configmap "kube-rbac-proxy" not found
Sep 07 11:24:15.162169 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:15.161850    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:15.162169 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:15.161899    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls podName:bcd1d32e-7641-4e8b-844b-9e8f1531ea8c nodeName:}" failed. No retries permitted until 2026-09-07 11:24:23.161886923 +0000 UTC m=+95.109237686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lks4" (UID: "bcd1d32e-7641-4e8b-844b-9e8f1531ea8c") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:24:17.885480 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:17.885445    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:17.886116 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:17.885551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:17.887831 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:17.887805    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"image-registry-59b7f6f5dd-5vs9k\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") " pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:17.887943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:17.887917    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fa80310-9705-4c3c-9c57-a1e8b46ffca8-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-n4g2g\" (UID: \"3fa80310-9705-4c3c-9c57-a1e8b46ffca8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:17.981887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:17.981852    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"
Sep 07 11:24:17.985624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:17.985608    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:18.260003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.259972    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g"]
Sep 07 11:24:18.263239 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.263214    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"]
Sep 07 11:24:18.266515 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:18.266484    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b56b1d_dabb_47f9_8bdb_6797d7dd3e55.slice/crio-767ce92ae8b19b03683e42bedb8a71f32d5ba270dd734d7b6897333e1bfb7fba WatchSource:0}: Error finding container 767ce92ae8b19b03683e42bedb8a71f32d5ba270dd734d7b6897333e1bfb7fba: Status 404 returned error can't find the container with id 767ce92ae8b19b03683e42bedb8a71f32d5ba270dd734d7b6897333e1bfb7fba
Sep 07 11:24:18.835607 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.835574    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g" event={"ID":"3fa80310-9705-4c3c-9c57-a1e8b46ffca8","Type":"ContainerStarted","Data":"e82ce62ed55934dd68b34c90e3e070dfe022127616f9d2e793a3cd0e53ad0318"}
Sep 07 11:24:18.837068 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.837041    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k" event={"ID":"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55","Type":"ContainerStarted","Data":"1f39ad5ba96baf6297f9005f7698c2ac6a3e8e9e458a9e9ab7f3710b3a5e20cc"}
Sep 07 11:24:18.837189 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.837088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k" event={"ID":"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55","Type":"ContainerStarted","Data":"767ce92ae8b19b03683e42bedb8a71f32d5ba270dd734d7b6897333e1bfb7fba"}
Sep 07 11:24:18.837250 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.837184    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:18.857432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:18.857382    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k" podStartSLOduration=16.857367677 podStartE2EDuration="16.857367677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:18.856668585 +0000 UTC m=+90.804019371" watchObservedRunningTime="2026-09-07 11:24:18.857367677 +0000 UTC m=+90.804718461"
Sep 07 11:24:19.397729 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:19.397692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:19.400001 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:19.399974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/151e67a0-7edf-4a1a-a0a0-d16fc7ded050-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-8lnf8\" (UID: \"151e67a0-7edf-4a1a-a0a0-d16fc7ded050\") " pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:19.577115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:19.577082    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8"
Sep 07 11:24:19.843274 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:19.843241    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-8lnf8"]
Sep 07 11:24:20.131704 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:20.131635    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151e67a0_7edf_4a1a_a0a0_d16fc7ded050.slice/crio-fe55e18ed8afe0d2b69c92ee535a61344d0e7ce807792394711dde72017b7736 WatchSource:0}: Error finding container fe55e18ed8afe0d2b69c92ee535a61344d0e7ce807792394711dde72017b7736: Status 404 returned error can't find the container with id fe55e18ed8afe0d2b69c92ee535a61344d0e7ce807792394711dde72017b7736
Sep 07 11:24:20.846755 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:20.846704    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g" event={"ID":"3fa80310-9705-4c3c-9c57-a1e8b46ffca8","Type":"ContainerStarted","Data":"68dc5bc92fa68636d7c7f802624a2be2060dcbc41180447accaf8ae0f8ac88a2"}
Sep 07 11:24:20.847207 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:20.846784    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g" event={"ID":"3fa80310-9705-4c3c-9c57-a1e8b46ffca8","Type":"ContainerStarted","Data":"5e013091d5f9f6613d8ed062144ce89549ab351ee448f732518296ec2abaf2c4"}
Sep 07 11:24:20.848220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:20.848191    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8" event={"ID":"151e67a0-7edf-4a1a-a0a0-d16fc7ded050","Type":"ContainerStarted","Data":"fe55e18ed8afe0d2b69c92ee535a61344d0e7ce807792394711dde72017b7736"}
Sep 07 11:24:20.862967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:20.862921    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-n4g2g" podStartSLOduration=17.010043148 podStartE2EDuration="18.862901172s" totalImagesPullingTime="1.852858024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:20.861838892 +0000 UTC m=+92.809189677" watchObservedRunningTime="2026-09-07 11:24:20.862901172 +0000 UTC m=+92.810251957"
Sep 07 11:24:21.853072 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:21.853036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8" event={"ID":"151e67a0-7edf-4a1a-a0a0-d16fc7ded050","Type":"ContainerStarted","Data":"148a76d8b14b5f3ba9c5cd629d0225d96eeb235064f9b2dd4e1e3dba4af80d02"}
Sep 07 11:24:21.868000 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:21.867955    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-8lnf8" podStartSLOduration=17.628327839 podStartE2EDuration="18.867942146s" totalImagesPullingTime="1.239614307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:21.867215744 +0000 UTC m=+93.814566530" watchObservedRunningTime="2026-09-07 11:24:21.867942146 +0000 UTC m=+93.815292930"
Sep 07 11:24:23.227439 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.227397    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:23.227439 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.227443    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:23.228089 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.228066    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:23.229704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.229686    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd1d32e-7641-4e8b-844b-9e8f1531ea8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lks4\" (UID: \"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c\") " pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:23.304867 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.304819    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4lks4"
Sep 07 11:24:23.575356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.575273    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4lks4"]
Sep 07 11:24:23.579184 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:23.579157    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd1d32e_7641_4e8b_844b_9e8f1531ea8c.slice/crio-46dc60f8cb170943859f6b2c7691ffa8c36cdce513b83c861fef795cbbc54482 WatchSource:0}: Error finding container 46dc60f8cb170943859f6b2c7691ffa8c36cdce513b83c861fef795cbbc54482: Status 404 returned error can't find the container with id 46dc60f8cb170943859f6b2c7691ffa8c36cdce513b83c861fef795cbbc54482
Sep 07 11:24:23.860969 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.860884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lks4" event={"ID":"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c","Type":"ContainerStarted","Data":"6d3d5a89c0eae892e184ae269a43ba713eb3112ae9255999590e8dbecd2e97da"}
Sep 07 11:24:23.860969 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:23.860948    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lks4" event={"ID":"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c","Type":"ContainerStarted","Data":"46dc60f8cb170943859f6b2c7691ffa8c36cdce513b83c861fef795cbbc54482"}
Sep 07 11:24:24.866935 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:24.866896    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lks4" event={"ID":"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c","Type":"ContainerStarted","Data":"140ff072617361179fd270678eedfa4b84747df317b05f930d8cb9666a4a6598"}
Sep 07 11:24:25.873951 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:25.873188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lks4" event={"ID":"bcd1d32e-7641-4e8b-844b-9e8f1531ea8c","Type":"ContainerStarted","Data":"4f314532bc4a87a0f2e5dfc763676d46c1a2702b2866f9446c41894bb2ae825c"}
Sep 07 11:24:25.893509 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:25.893463    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4lks4" podStartSLOduration=16.916183902 podStartE2EDuration="18.893448855s" totalImagesPullingTime="1.977264953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:24:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:25.893156387 +0000 UTC m=+97.840507170" watchObservedRunningTime="2026-09-07 11:24:25.893448855 +0000 UTC m=+97.840799639"
Sep 07 11:24:26.350030 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:26.349993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:24:26.352272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:26.352252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84e6938a-fe71-410a-aa30-9a929b1be9a9-metrics-tls\") pod \"dns-default-q7r6q\" (UID: \"84e6938a-fe71-410a-aa30-9a929b1be9a9\") " pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:24:26.361456 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:26.361436    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ph89j\""
Sep 07 11:24:26.370165 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:26.370145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:24:26.645297 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:26.645219    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q7r6q"]
Sep 07 11:24:26.648066 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:26.648031    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e6938a_fe71_410a_aa30_9a929b1be9a9.slice/crio-c7fd49a9e08cd45990f5ebb132f427349c9dd8b79114d68a65e6d19316c9e4c3 WatchSource:0}: Error finding container c7fd49a9e08cd45990f5ebb132f427349c9dd8b79114d68a65e6d19316c9e4c3: Status 404 returned error can't find the container with id c7fd49a9e08cd45990f5ebb132f427349c9dd8b79114d68a65e6d19316c9e4c3
Sep 07 11:24:26.877197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:26.877158    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7r6q" event={"ID":"84e6938a-fe71-410a-aa30-9a929b1be9a9","Type":"ContainerStarted","Data":"c7fd49a9e08cd45990f5ebb132f427349c9dd8b79114d68a65e6d19316c9e4c3"}
Sep 07 11:24:28.886486 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:28.886448    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7r6q" event={"ID":"84e6938a-fe71-410a-aa30-9a929b1be9a9","Type":"ContainerStarted","Data":"3ef43e937097ae0540070ebdb52eca735af4106df4bbc0eb1c10ab1f48e4903a"}
Sep 07 11:24:28.886855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:28.886510    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7r6q" event={"ID":"84e6938a-fe71-410a-aa30-9a929b1be9a9","Type":"ContainerStarted","Data":"705ff34abf28e0a6d1b75506b3667f738ae88ef2701b8a9bb90d04d532a7e63e"}
Sep 07 11:24:28.886855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:28.886551    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:24:28.904496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:28.904452    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q7r6q" podStartSLOduration=65.495543674 podStartE2EDuration="1m6.904438367s" totalImagesPullingTime="1.408894693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:28.903189841 +0000 UTC m=+100.850540627" watchObservedRunningTime="2026-09-07 11:24:28.904438367 +0000 UTC m=+100.851789150"
Sep 07 11:24:29.751921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.751881    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"]
Sep 07 11:24:29.781880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.781852    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"]
Sep 07 11:24:29.782021 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.781889    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"]
Sep 07 11:24:29.782021 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.781960    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.782021 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.781980    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7968944dff-4prpl"]
Sep 07 11:24:29.784352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.784319    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 07 11:24:29.784477 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.784378    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 07 11:24:29.784477 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.784381    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 07 11:24:29.784477 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.784429    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 07 11:24:29.805852 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.805830    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7968944dff-4prpl"]
Sep 07 11:24:29.805948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.805897    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877035 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1131812-e49c-4586-98d5-efbedc41302d-installation-pull-secrets\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877165 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877052    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7l2w\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-kube-api-access-k7l2w\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877165 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877072    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-registry-tls\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877165 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7sp\" (UniqueName: \"kubernetes.io/projected/c4d19bce-2c8f-493a-95bd-9f798a31463a-kube-api-access-tg7sp\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.877165 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1131812-e49c-4586-98d5-efbedc41302d-ca-trust-extracted\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877178    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-bound-sa-token\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877234    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1131812-e49c-4586-98d5-efbedc41302d-image-registry-private-configuration\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877263    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d19bce-2c8f-493a-95bd-9f798a31463a-tmp\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.877316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877280    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1131812-e49c-4586-98d5-efbedc41302d-trusted-ca\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.877316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c4d19bce-2c8f-493a-95bd-9f798a31463a-klusterlet-config\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.877500 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.877324    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1131812-e49c-4586-98d5-efbedc41302d-registry-certificates\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.977965 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.977885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1131812-e49c-4586-98d5-efbedc41302d-installation-pull-secrets\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.977965 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.977963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7l2w\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-kube-api-access-k7l2w\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-registry-tls\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7sp\" (UniqueName: \"kubernetes.io/projected/c4d19bce-2c8f-493a-95bd-9f798a31463a-kube-api-access-tg7sp\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978078    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1131812-e49c-4586-98d5-efbedc41302d-ca-trust-extracted\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-bound-sa-token\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978128    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1131812-e49c-4586-98d5-efbedc41302d-image-registry-private-configuration\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978187    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d19bce-2c8f-493a-95bd-9f798a31463a-tmp\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978225    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1131812-e49c-4586-98d5-efbedc41302d-trusted-ca\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978276    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c4d19bce-2c8f-493a-95bd-9f798a31463a-klusterlet-config\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.978414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1131812-e49c-4586-98d5-efbedc41302d-registry-certificates\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.978849 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.978834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d19bce-2c8f-493a-95bd-9f798a31463a-tmp\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.979723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.979649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1131812-e49c-4586-98d5-efbedc41302d-ca-trust-extracted\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.979878 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.979771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1131812-e49c-4586-98d5-efbedc41302d-trusted-ca\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.980001 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.979981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1131812-e49c-4586-98d5-efbedc41302d-registry-certificates\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.981177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.981152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1131812-e49c-4586-98d5-efbedc41302d-image-registry-private-configuration\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.981455 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.981426    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1131812-e49c-4586-98d5-efbedc41302d-installation-pull-secrets\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.981528 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.981500    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c4d19bce-2c8f-493a-95bd-9f798a31463a-klusterlet-config\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.981578 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.981564    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-registry-tls\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.987149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.987109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-bound-sa-token\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:29.987348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.987330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7sp\" (UniqueName: \"kubernetes.io/projected/c4d19bce-2c8f-493a-95bd-9f798a31463a-kube-api-access-tg7sp\") pod \"klusterlet-addon-workmgr-745b99675f-9bw8k\" (UID: \"c4d19bce-2c8f-493a-95bd-9f798a31463a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:29.987582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:29.987561    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7l2w\" (UniqueName: \"kubernetes.io/projected/f1131812-e49c-4586-98d5-efbedc41302d-kube-api-access-k7l2w\") pod \"image-registry-7968944dff-4prpl\" (UID: \"f1131812-e49c-4586-98d5-efbedc41302d\") " pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:30.091101 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.091020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:30.114347 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.114324    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:30.389347 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.389156    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"]
Sep 07 11:24:30.392135 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:30.392083    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d19bce_2c8f_493a_95bd_9f798a31463a.slice/crio-06b1ed0c6953529b85cd75f422c72e178e67823be8f7cf58814c1cfd066ebe60 WatchSource:0}: Error finding container 06b1ed0c6953529b85cd75f422c72e178e67823be8f7cf58814c1cfd066ebe60: Status 404 returned error can't find the container with id 06b1ed0c6953529b85cd75f422c72e178e67823be8f7cf58814c1cfd066ebe60
Sep 07 11:24:30.418102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.418073    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7968944dff-4prpl"]
Sep 07 11:24:30.420723 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:30.420699    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1131812_e49c_4586_98d5_efbedc41302d.slice/crio-46dc410d5dd9d462b88149dafe1344c654a6d0c356dd326ba28d5994bcb35076 WatchSource:0}: Error finding container 46dc410d5dd9d462b88149dafe1344c654a6d0c356dd326ba28d5994bcb35076: Status 404 returned error can't find the container with id 46dc410d5dd9d462b88149dafe1344c654a6d0c356dd326ba28d5994bcb35076
Sep 07 11:24:30.779806 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.779774    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6628l"
Sep 07 11:24:30.894703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.894665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k" event={"ID":"c4d19bce-2c8f-493a-95bd-9f798a31463a","Type":"ContainerStarted","Data":"06b1ed0c6953529b85cd75f422c72e178e67823be8f7cf58814c1cfd066ebe60"}
Sep 07 11:24:30.896294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.896261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7968944dff-4prpl" event={"ID":"f1131812-e49c-4586-98d5-efbedc41302d","Type":"ContainerStarted","Data":"f51b5eb7753e98454e99c5823eb130305b57eb0b36a2e23f74bc7848a837db19"}
Sep 07 11:24:30.896405 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.896300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7968944dff-4prpl" event={"ID":"f1131812-e49c-4586-98d5-efbedc41302d","Type":"ContainerStarted","Data":"46dc410d5dd9d462b88149dafe1344c654a6d0c356dd326ba28d5994bcb35076"}
Sep 07 11:24:30.896443 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.896402    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:30.931819 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:30.931774    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7968944dff-4prpl" podStartSLOduration=1.931758146 podStartE2EDuration="1.931758146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:30.929169591 +0000 UTC m=+102.876520374" watchObservedRunningTime="2026-09-07 11:24:30.931758146 +0000 UTC m=+102.879108920"
Sep 07 11:24:34.915483 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.915438    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k" event={"ID":"c4d19bce-2c8f-493a-95bd-9f798a31463a","Type":"ContainerStarted","Data":"21e44ff9b8deec9c9ea7e70c45ac923d1b18dc75f8d4f264f79be692be2fe0ca"}
Sep 07 11:24:34.915998 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.915638    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:34.917496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.917467    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k"
Sep 07 11:24:34.933527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.933470    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-745b99675f-9bw8k" podStartSLOduration=2.22073281 podStartE2EDuration="5.933454421s" totalImagesPullingTime="3.712721611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:34.932041911 +0000 UTC m=+106.879392695" watchObservedRunningTime="2026-09-07 11:24:34.933454421 +0000 UTC m=+106.880805204"
Sep 07 11:24:34.975902 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.975865    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9688ccf8b-tzk5b"]
Sep 07 11:24:34.977905 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.977888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:34.980250 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980221    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 07 11:24:34.980389 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980297    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 07 11:24:34.980389 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980319    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 07 11:24:34.980517 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980439    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k84wv\""
Sep 07 11:24:34.980554 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980540    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 07 11:24:34.980802 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980783    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 07 11:24:34.980860 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980786    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 07 11:24:34.980860 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.980791    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 07 11:24:34.991555 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:34.991533    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9688ccf8b-tzk5b"]
Sep 07 11:24:35.118885 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.118830    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-oauth-config\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.119075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.118993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sdpq\" (UniqueName: \"kubernetes.io/projected/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-kube-api-access-8sdpq\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.119075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.119027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-serving-cert\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.119075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.119053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-config\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.119194 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.119114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-service-ca\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.119194 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.119146    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-oauth-serving-cert\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.220424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.220315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdpq\" (UniqueName: \"kubernetes.io/projected/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-kube-api-access-8sdpq\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.220424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.220373    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-serving-cert\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.220424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.220398    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-config\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.220424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.220427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-service-ca\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.220760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.220444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-oauth-serving-cert\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.220760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.220466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-oauth-config\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.221177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.221148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-config\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.221177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.221167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-service-ca\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.221376 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.221356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-oauth-serving-cert\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.223006 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.222987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-oauth-config\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.223006 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.223000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-serving-cert\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.230124 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.230101    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sdpq\" (UniqueName: \"kubernetes.io/projected/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-kube-api-access-8sdpq\") pod \"console-9688ccf8b-tzk5b\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") " pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.287151 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.287112    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:35.559204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.559168    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9688ccf8b-tzk5b"]
Sep 07 11:24:35.563067 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:35.563045    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98d02bb_f2c9_4e73_8ac4_1de26f0613d5.slice/crio-6486950ddb0eda910d7528ad236666785536a54c209ea562a046792ab1b78d60 WatchSource:0}: Error finding container 6486950ddb0eda910d7528ad236666785536a54c209ea562a046792ab1b78d60: Status 404 returned error can't find the container with id 6486950ddb0eda910d7528ad236666785536a54c209ea562a046792ab1b78d60
Sep 07 11:24:35.919839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:35.919728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9688ccf8b-tzk5b" event={"ID":"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5","Type":"ContainerStarted","Data":"6486950ddb0eda910d7528ad236666785536a54c209ea562a046792ab1b78d60"}
Sep 07 11:24:38.891672 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:38.891641    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q7r6q"
Sep 07 11:24:38.933110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:38.933079    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9688ccf8b-tzk5b" event={"ID":"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5","Type":"ContainerStarted","Data":"df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65"}
Sep 07 11:24:38.950933 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:38.950887    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9688ccf8b-tzk5b" podStartSLOduration=2.241192458 podStartE2EDuration="4.950870918s" totalImagesPullingTime="2.70967846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:38.9496452 +0000 UTC m=+110.896995996" watchObservedRunningTime="2026-09-07 11:24:38.950870918 +0000 UTC m=+110.898221749"
Sep 07 11:24:39.787853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:39.787825    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:45.287421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:45.287389    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:45.287861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:45.287440    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:45.293075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:45.293052    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:45.958891 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:45.958865    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:24:51.908161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:51.908124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7968944dff-4prpl"
Sep 07 11:24:52.969029 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.968996    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-87lq8"]
Sep 07 11:24:52.973914 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.973893    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:52.976063 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.976041    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 11:24:52.976212 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.976192    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wb972\""
Sep 07 11:24:52.976262 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.976226    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 11:24:52.976778 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.976761    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 11:24:52.976864 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.976824    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 11:24:52.977080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.977057    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 11:24:52.977080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:52.977069    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 11:24:53.059606 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e7116d-8c77-473c-95ed-04e86c9e7705-metrics-client-ca\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059614    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-tls\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059637    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-root\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059656    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-accelerators-collector-config\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-sys\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxkd\" (UniqueName: \"kubernetes.io/projected/60e7116d-8c77-473c-95ed-04e86c9e7705-kube-api-access-6qxkd\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.059919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059871    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-wtmp\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.060050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.059938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-textfile\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.160934 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.160910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.160938    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-accelerators-collector-config\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.160962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-sys\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.160997    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxkd\" (UniqueName: \"kubernetes.io/projected/60e7116d-8c77-473c-95ed-04e86c9e7705-kube-api-access-6qxkd\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161016    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-wtmp\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161038    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-textfile\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-sys\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161066    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e7116d-8c77-473c-95ed-04e86c9e7705-metrics-client-ca\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-tls\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-root\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161175    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-wtmp\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161316 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161189    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/60e7116d-8c77-473c-95ed-04e86c9e7705-root\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161316 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:53.161264    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 07 11:24:53.161658 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:24:53.161341    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-tls podName:60e7116d-8c77-473c-95ed-04e86c9e7705 nodeName:}" failed. No retries permitted until 2026-09-07 11:24:53.661320373 +0000 UTC m=+125.608671148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-tls") pod "node-exporter-87lq8" (UID: "60e7116d-8c77-473c-95ed-04e86c9e7705") : secret "node-exporter-tls" not found
Sep 07 11:24:53.161658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-textfile\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-accelerators-collector-config\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.161658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.161615    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e7116d-8c77-473c-95ed-04e86c9e7705-metrics-client-ca\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.163498 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.163478    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.170883 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.170863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxkd\" (UniqueName: \"kubernetes.io/projected/60e7116d-8c77-473c-95ed-04e86c9e7705-kube-api-access-6qxkd\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.663942 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.663911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-tls\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.666158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.666135    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/60e7116d-8c77-473c-95ed-04e86c9e7705-node-exporter-tls\") pod \"node-exporter-87lq8\" (UID: \"60e7116d-8c77-473c-95ed-04e86c9e7705\") " pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.883502 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.883466    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-87lq8"
Sep 07 11:24:53.900810 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:53.900786    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e7116d_8c77_473c_95ed_04e86c9e7705.slice/crio-be587d8907a4d6332826b5fa6fe869039b87199fedf9d892a49160b99e5edbbb WatchSource:0}: Error finding container be587d8907a4d6332826b5fa6fe869039b87199fedf9d892a49160b99e5edbbb: Status 404 returned error can't find the container with id be587d8907a4d6332826b5fa6fe869039b87199fedf9d892a49160b99e5edbbb
Sep 07 11:24:53.984854 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:53.984823    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87lq8" event={"ID":"60e7116d-8c77-473c-95ed-04e86c9e7705","Type":"ContainerStarted","Data":"be587d8907a4d6332826b5fa6fe869039b87199fedf9d892a49160b99e5edbbb"}
Sep 07 11:24:54.802038 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:54.801995    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k" podUID="e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" containerName="registry" containerID="cri-o://1f39ad5ba96baf6297f9005f7698c2ac6a3e8e9e458a9e9ab7f3710b3a5e20cc" gracePeriod=30
Sep 07 11:24:54.989362 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:54.989331    2560 generic.go:356] "Generic (PLEG): container finished" podID="e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" containerID="1f39ad5ba96baf6297f9005f7698c2ac6a3e8e9e458a9e9ab7f3710b3a5e20cc" exitCode=0
Sep 07 11:24:54.989680 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:54.989372    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k" event={"ID":"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55","Type":"ContainerDied","Data":"1f39ad5ba96baf6297f9005f7698c2ac6a3e8e9e458a9e9ab7f3710b3a5e20cc"}
Sep 07 11:24:55.100462 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.100438    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:55.274290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274258    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274307    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-certificates\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274332    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-trusted-ca\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274380    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmnwd\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-kube-api-access-tmnwd\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274410    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-installation-pull-secrets\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274427    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-image-registry-private-configuration\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274699 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274463    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-ca-trust-extracted\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274699 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274504    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-bound-sa-token\") pod \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\" (UID: \"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55\") "
Sep 07 11:24:55.274832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274774    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:24:55.274949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.274909    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:24:55.276894 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.276859    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-kube-api-access-tmnwd" (OuterVolumeSpecName: "kube-api-access-tmnwd") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "kube-api-access-tmnwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:24:55.276894 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.276877    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:24:55.277033 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.276961    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:24:55.277033 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.277022    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:24:55.277234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.277209    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:24:55.282717 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.282695    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" (UID: "e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:24:55.375952 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375889    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-certificates\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.375952 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375915    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-trusted-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.375952 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375925    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmnwd\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-kube-api-access-tmnwd\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.375952 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375934    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-installation-pull-secrets\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.375952 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375944    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-image-registry-private-configuration\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.375952 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375953    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-ca-trust-extracted\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.376199 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375962    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-bound-sa-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.376199 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.375970    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55-registry-tls\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:24:55.994324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.994288    2560 generic.go:356] "Generic (PLEG): container finished" podID="60e7116d-8c77-473c-95ed-04e86c9e7705" containerID="6dbabc2f1040487229daf05249637dce103dc92ced73b6ad46461211f35ffd2d" exitCode=0
Sep 07 11:24:55.994683 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.994374    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87lq8" event={"ID":"60e7116d-8c77-473c-95ed-04e86c9e7705","Type":"ContainerDied","Data":"6dbabc2f1040487229daf05249637dce103dc92ced73b6ad46461211f35ffd2d"}
Sep 07 11:24:55.995834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.995809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k" event={"ID":"e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55","Type":"ContainerDied","Data":"767ce92ae8b19b03683e42bedb8a71f32d5ba270dd734d7b6897333e1bfb7fba"}
Sep 07 11:24:55.995834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.995826    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"
Sep 07 11:24:55.995954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:55.995873    2560 scope.go:117] "RemoveContainer" containerID="1f39ad5ba96baf6297f9005f7698c2ac6a3e8e9e458a9e9ab7f3710b3a5e20cc"
Sep 07 11:24:56.025819 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:56.025797    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"]
Sep 07 11:24:56.032399 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:56.032378    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59b7f6f5dd-5vs9k"]
Sep 07 11:24:56.496170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:56.496137    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" path="/var/lib/kubelet/pods/e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55/volumes"
Sep 07 11:24:57.002056 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.002020    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87lq8" event={"ID":"60e7116d-8c77-473c-95ed-04e86c9e7705","Type":"ContainerStarted","Data":"a537caf207de0b33c7593dd3e66b69e5de6d68a1878d2adbac7c16334aaed900"}
Sep 07 11:24:57.002398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.002076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87lq8" event={"ID":"60e7116d-8c77-473c-95ed-04e86c9e7705","Type":"ContainerStarted","Data":"6dbe2d5ea06e22e150ed1f362a816b849d9b759525688cab11a07c18a209b38d"}
Sep 07 11:24:57.020724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.020681    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-87lq8" podStartSLOduration=3.848156567 podStartE2EDuration="5.020666232s" totalImagesPullingTime="1.109527684s" totalInitContainerRuntime="62.981981ms" isStatefulPod=true podCreationTimestamp="2026-09-07 11:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:57.019172381 +0000 UTC m=+128.966523166" watchObservedRunningTime="2026-09-07 11:24:57.020666232 +0000 UTC m=+128.968017016"
Sep 07 11:24:57.788587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.788554    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6658b8986f-k24qn"]
Sep 07 11:24:57.788811 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.788799    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" containerName="registry"
Sep 07 11:24:57.788859 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.788812    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" containerName="registry"
Sep 07 11:24:57.788892 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.788859    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b56b1d-dabb-47f9-8bdb-6797d7dd3e55" containerName="registry"
Sep 07 11:24:57.791871 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.791853    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.798156 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.798137    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 07 11:24:57.807669 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.807647    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6658b8986f-k24qn"]
Sep 07 11:24:57.892346 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6mfn\" (UniqueName: \"kubernetes.io/projected/8f6469c2-c37f-4114-b026-40c2e1c35860-kube-api-access-w6mfn\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.892444 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-service-ca\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.892444 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-serving-cert\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.892556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892450    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-trusted-ca-bundle\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.892556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-oauth-config\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.892556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892507    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-console-config\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.892556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.892524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-oauth-serving-cert\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.993798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6mfn\" (UniqueName: \"kubernetes.io/projected/8f6469c2-c37f-4114-b026-40c2e1c35860-kube-api-access-w6mfn\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.993916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-service-ca\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.993916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-serving-cert\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.993916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993891    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-trusted-ca-bundle\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.994055 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-oauth-config\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.994055 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-console-config\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.994055 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.993971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-oauth-serving-cert\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.994764 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.994721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-service-ca\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.994764 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.994783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-console-config\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.994974 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.994784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-oauth-serving-cert\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.995352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.995328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-trusted-ca-bundle\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.996341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.996316    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-serving-cert\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:57.996421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:57.996355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-oauth-config\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:58.001880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.001854    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6mfn\" (UniqueName: \"kubernetes.io/projected/8f6469c2-c37f-4114-b026-40c2e1c35860-kube-api-access-w6mfn\") pod \"console-6658b8986f-k24qn\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") " pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:58.100504 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.100426    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:24:58.295604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.295564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:24:58.297700 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.297670    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e39c68a-5691-47a5-bb3c-fa289e615054-metrics-certs\") pod \"network-metrics-daemon-bjfn8\" (UID: \"9e39c68a-5691-47a5-bb3c-fa289e615054\") " pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:24:58.308465 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.308434    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qlhnv\""
Sep 07 11:24:58.316971 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.316958    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bjfn8"
Sep 07 11:24:58.369687 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.369585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6658b8986f-k24qn"]
Sep 07 11:24:58.372816 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:58.372784    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6469c2_c37f_4114_b026_40c2e1c35860.slice/crio-29a2b9835c16e4c0634280697b2e761258c005a300fabbbb8f03703291a935fd WatchSource:0}: Error finding container 29a2b9835c16e4c0634280697b2e761258c005a300fabbbb8f03703291a935fd: Status 404 returned error can't find the container with id 29a2b9835c16e4c0634280697b2e761258c005a300fabbbb8f03703291a935fd
Sep 07 11:24:58.588597 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:58.588566    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bjfn8"]
Sep 07 11:24:58.591428 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:58.591401    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e39c68a_5691_47a5_bb3c_fa289e615054.slice/crio-016510e07fba322682025d1d2cbc69f6de341e2c8889efd25715b81f96cf6b2e WatchSource:0}: Error finding container 016510e07fba322682025d1d2cbc69f6de341e2c8889efd25715b81f96cf6b2e: Status 404 returned error can't find the container with id 016510e07fba322682025d1d2cbc69f6de341e2c8889efd25715b81f96cf6b2e
Sep 07 11:24:59.010377 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.010330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bjfn8" event={"ID":"9e39c68a-5691-47a5-bb3c-fa289e615054","Type":"ContainerStarted","Data":"016510e07fba322682025d1d2cbc69f6de341e2c8889efd25715b81f96cf6b2e"}
Sep 07 11:24:59.011782 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.011757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6658b8986f-k24qn" event={"ID":"8f6469c2-c37f-4114-b026-40c2e1c35860","Type":"ContainerStarted","Data":"d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d"}
Sep 07 11:24:59.011875 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.011798    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6658b8986f-k24qn" event={"ID":"8f6469c2-c37f-4114-b026-40c2e1c35860","Type":"ContainerStarted","Data":"29a2b9835c16e4c0634280697b2e761258c005a300fabbbb8f03703291a935fd"}
Sep 07 11:24:59.193278 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.193219    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6658b8986f-k24qn" podStartSLOduration=2.193200241 podStartE2EDuration="2.193200241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:24:59.029312935 +0000 UTC m=+130.976663719" watchObservedRunningTime="2026-09-07 11:24:59.193200241 +0000 UTC m=+131.140551024"
Sep 07 11:24:59.193667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.193612    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:24:59.197864 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.197843    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.200409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200236    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 11:24:59.200409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200366    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 11:24:59.200409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200382    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9skmg\""
Sep 07 11:24:59.200673 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200655    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 11:24:59.200808 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200790    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 11:24:59.200953 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200916    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 11:24:59.201080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201021    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 11:24:59.201154 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201137    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 11:24:59.201154 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.200363    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 11:24:59.201279 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201216    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 11:24:59.201847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201476    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1v6tj0nplhq81\""
Sep 07 11:24:59.201847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201494    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 11:24:59.201847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201500    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 11:24:59.201847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.201690    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 07 11:24:59.202453 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.202359    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 11:24:59.204287 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.204266    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 11:24:59.207729 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.207434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 11:24:59.211695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.211672    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:24:59.303216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303135    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/75ca6d96-a319-483a-bfd2-aade442e5022-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303179    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-web-config\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303268    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303306    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303362    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.303664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.303603    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-config\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.304112 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.304087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.304185 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.304159    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/75ca6d96-a319-483a-bfd2-aade442e5022-config-out\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.304244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.304186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.304244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.304220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnnt\" (UniqueName: \"kubernetes.io/projected/75ca6d96-a319-483a-bfd2-aade442e5022-kube-api-access-qfnnt\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.304346 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.304273    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.405708 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.405664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.405708 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.405709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-config\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.405962 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.405733    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.405962 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.405783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/75ca6d96-a319-483a-bfd2-aade442e5022-config-out\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnnt\" (UniqueName: \"kubernetes.io/projected/75ca6d96-a319-483a-bfd2-aade442e5022-kube-api-access-qfnnt\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/75ca6d96-a319-483a-bfd2-aade442e5022-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-web-config\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406340    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406379    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.406936    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407539    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407771    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.407887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.407830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.408531 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.408500    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.409561 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.409531    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/75ca6d96-a319-483a-bfd2-aade442e5022-config-out\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.409666 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.409646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.409729 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.409718    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-web-config\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.409944 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.409919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.410277 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.410252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.410927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.410769    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.410927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.410855    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.411236 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.411212    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-config\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.411322 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.411223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.412890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.412866    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/75ca6d96-a319-483a-bfd2-aade442e5022-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.413048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.412983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.413245 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.413196    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/75ca6d96-a319-483a-bfd2-aade442e5022-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.413336 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.413235    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.413573 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.413534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.413659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.413640    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/75ca6d96-a319-483a-bfd2-aade442e5022-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.415076 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.415058    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnnt\" (UniqueName: \"kubernetes.io/projected/75ca6d96-a319-483a-bfd2-aade442e5022-kube-api-access-qfnnt\") pod \"prometheus-k8s-0\" (UID: \"75ca6d96-a319-483a-bfd2-aade442e5022\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.510477 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.510445    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:24:59.841248 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:24:59.841165    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:24:59.843941 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:24:59.843917    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ca6d96_a319_483a_bfd2_aade442e5022.slice/crio-b6d77a9f91123f449447b5fb2e86661de5e695cd51415f5fa20efbb7661ed84e WatchSource:0}: Error finding container b6d77a9f91123f449447b5fb2e86661de5e695cd51415f5fa20efbb7661ed84e: Status 404 returned error can't find the container with id b6d77a9f91123f449447b5fb2e86661de5e695cd51415f5fa20efbb7661ed84e
Sep 07 11:25:00.017151 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:00.017111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bjfn8" event={"ID":"9e39c68a-5691-47a5-bb3c-fa289e615054","Type":"ContainerStarted","Data":"c1bc3778a37ed67ba81063910b84799d46af366224ce053f45bf322b5d93726f"}
Sep 07 11:25:00.017340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:00.017180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bjfn8" event={"ID":"9e39c68a-5691-47a5-bb3c-fa289e615054","Type":"ContainerStarted","Data":"2259d74a5a9102db0960fbc817065b1c91227732c48966266b648099a535c3ab"}
Sep 07 11:25:00.018314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:00.018280    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"b6d77a9f91123f449447b5fb2e86661de5e695cd51415f5fa20efbb7661ed84e"}
Sep 07 11:25:00.032791 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:00.032723    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bjfn8" podStartSLOduration=131.008277666 podStartE2EDuration="2m12.032707552s" totalImagesPullingTime="1.024429886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:25:00.031854474 +0000 UTC m=+131.979205258" watchObservedRunningTime="2026-09-07 11:25:00.032707552 +0000 UTC m=+131.980058335"
Sep 07 11:25:02.029392 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:02.029354    2560 generic.go:356] "Generic (PLEG): container finished" podID="75ca6d96-a319-483a-bfd2-aade442e5022" containerID="3e80b2a77a0b3220361aa30b5d4845ccc5c47c5d5ca5b7e19934dffc8644c762" exitCode=0
Sep 07 11:25:02.029886 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:02.029442    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerDied","Data":"3e80b2a77a0b3220361aa30b5d4845ccc5c47c5d5ca5b7e19934dffc8644c762"}
Sep 07 11:25:04.705853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.705815    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6658b8986f-k24qn"]
Sep 07 11:25:04.766523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.766485    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76bbf9b6d5-fhvhj"]
Sep 07 11:25:04.772977 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.772946    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.778716 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.778690    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76bbf9b6d5-fhvhj"]
Sep 07 11:25:04.853094 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-service-ca\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.853283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853105    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-oauth-config\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.853283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853176    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-trusted-ca-bundle\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.853283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw4rr\" (UniqueName: \"kubernetes.io/projected/84050e73-55c7-4cad-9752-c13a36c4c6b5-kube-api-access-dw4rr\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.853283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853263    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-serving-cert\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.853467 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-oauth-serving-cert\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.853467 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.853343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-config\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954066 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-config\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954066 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-service-ca\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954315 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-oauth-config\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954315 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-trusted-ca-bundle\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954315 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954270    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw4rr\" (UniqueName: \"kubernetes.io/projected/84050e73-55c7-4cad-9752-c13a36c4c6b5-kube-api-access-dw4rr\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954490 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-serving-cert\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954490 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-oauth-serving-cert\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.954904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954880    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-service-ca\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.955020 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.954919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-config\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.955448 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.955420    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-trusted-ca-bundle\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.955561 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.955460    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-oauth-serving-cert\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.957128 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.957047    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-oauth-config\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.957346 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.957327    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-serving-cert\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:04.961451 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:04.961427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw4rr\" (UniqueName: \"kubernetes.io/projected/84050e73-55c7-4cad-9752-c13a36c4c6b5-kube-api-access-dw4rr\") pod \"console-76bbf9b6d5-fhvhj\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") " pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:05.085412 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:05.085372    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:05.514311 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:05.514284    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76bbf9b6d5-fhvhj"]
Sep 07 11:25:05.517381 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:25:05.517359    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84050e73_55c7_4cad_9752_c13a36c4c6b5.slice/crio-63d406c52119e6e7eac42ea8875e66346586354d7391fa8bbbfb65217298de5c WatchSource:0}: Error finding container 63d406c52119e6e7eac42ea8875e66346586354d7391fa8bbbfb65217298de5c: Status 404 returned error can't find the container with id 63d406c52119e6e7eac42ea8875e66346586354d7391fa8bbbfb65217298de5c
Sep 07 11:25:06.048661 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:06.048628    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"1aeb647a425338e772bc05c81f21ba134497611ecbf1af725a97f7ccceb86e39"}
Sep 07 11:25:06.049166 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:06.048697    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"66fd0b7895d124181330eb1b4c08e716be5abf1a580aa900ae9e918614d6df50"}
Sep 07 11:25:06.050143 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:06.050120    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bbf9b6d5-fhvhj" event={"ID":"84050e73-55c7-4cad-9752-c13a36c4c6b5","Type":"ContainerStarted","Data":"85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a"}
Sep 07 11:25:06.050250 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:06.050164    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bbf9b6d5-fhvhj" event={"ID":"84050e73-55c7-4cad-9752-c13a36c4c6b5","Type":"ContainerStarted","Data":"63d406c52119e6e7eac42ea8875e66346586354d7391fa8bbbfb65217298de5c"}
Sep 07 11:25:06.068482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:06.068441    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76bbf9b6d5-fhvhj" podStartSLOduration=2.068426802 podStartE2EDuration="2.068426802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:25:06.066874609 +0000 UTC m=+138.014225394" watchObservedRunningTime="2026-09-07 11:25:06.068426802 +0000 UTC m=+138.015777656"
Sep 07 11:25:08.065094 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:08.065056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"799d62aab4632269ca5f0f6d405deb861a278c4da20628c169e8e773c58de429"}
Sep 07 11:25:08.065572 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:08.065153    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"6ca4d84e8f57900937bfe34313d1ce446e6e449763d665bf46cbd4428ce3cb5d"}
Sep 07 11:25:08.065572 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:08.065179    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"97b30a2d9a58cd35f70104ff9db79365c0c468b495d7f9490daac6f0b98deb0b"}
Sep 07 11:25:08.065572 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:08.065207    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"75ca6d96-a319-483a-bfd2-aade442e5022","Type":"ContainerStarted","Data":"6025198c5b36be47b812b30c36972793e201e15559f0459d1e0dc1e91202bb20"}
Sep 07 11:25:08.091854 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:08.091795    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.738115764 podStartE2EDuration="9.091733302s" totalImagesPullingTime="6.335418136s" totalInitContainerRuntime="18.199402ms" isStatefulPod=false podCreationTimestamp="2026-09-07 11:24:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:25:08.091316157 +0000 UTC m=+140.038666942" watchObservedRunningTime="2026-09-07 11:25:08.091733302 +0000 UTC m=+140.039084127"
Sep 07 11:25:08.101281 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:08.101254    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:25:09.511816 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:09.511784    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:25:15.085882 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:15.085842    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:15.086283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:15.085937    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:15.090511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:15.090484    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:15.097218 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:15.097200    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:25:15.151908 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:15.151873    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9688ccf8b-tzk5b"]
Sep 07 11:25:29.727497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:29.727437    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6658b8986f-k24qn" podUID="8f6469c2-c37f-4114-b026-40c2e1c35860" containerName="console" containerID="cri-o://d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d" gracePeriod=15
Sep 07 11:25:29.960245 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:29.960222    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:25:30.070317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070225    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6mfn\" (UniqueName: \"kubernetes.io/projected/8f6469c2-c37f-4114-b026-40c2e1c35860-kube-api-access-w6mfn\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070270    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-oauth-config\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070296    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-trusted-ca-bundle\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070317    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-service-ca\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070338    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-oauth-serving-cert\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070368    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-serving-cert\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070382    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-console-config\") pod \"8f6469c2-c37f-4114-b026-40c2e1c35860\" (UID: \"8f6469c2-c37f-4114-b026-40c2e1c35860\") "
Sep 07 11:25:30.070860 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070838    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:30.070947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070911    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:30.071007 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.070918    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-service-ca" (OuterVolumeSpecName: "service-ca") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:30.071062 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.071029    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-console-config" (OuterVolumeSpecName: "console-config") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:30.072618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.072592    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:25:30.072727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.072624    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f6469c2-c37f-4114-b026-40c2e1c35860-kube-api-access-w6mfn" (OuterVolumeSpecName: "kube-api-access-w6mfn") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "kube-api-access-w6mfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:25:30.072727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.072636    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8f6469c2-c37f-4114-b026-40c2e1c35860" (UID: "8f6469c2-c37f-4114-b026-40c2e1c35860"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:25:30.150889 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.150851    2560 generic.go:356] "Generic (PLEG): container finished" podID="8f6469c2-c37f-4114-b026-40c2e1c35860" containerID="d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d" exitCode=0
Sep 07 11:25:30.151044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.150901    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6658b8986f-k24qn"
Sep 07 11:25:30.151044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.150933    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6658b8986f-k24qn" event={"ID":"8f6469c2-c37f-4114-b026-40c2e1c35860","Type":"ContainerDied","Data":"d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d"}
Sep 07 11:25:30.151044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.150992    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6658b8986f-k24qn" event={"ID":"8f6469c2-c37f-4114-b026-40c2e1c35860","Type":"ContainerDied","Data":"29a2b9835c16e4c0634280697b2e761258c005a300fabbbb8f03703291a935fd"}
Sep 07 11:25:30.151044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.151013    2560 scope.go:117] "RemoveContainer" containerID="d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d"
Sep 07 11:25:30.160964 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.160657    2560 scope.go:117] "RemoveContainer" containerID="d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d"
Sep 07 11:25:30.161143 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:25:30.161115    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d\": container with ID starting with d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d not found: ID does not exist" containerID="d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d"
Sep 07 11:25:30.161218 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.161155    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d"} err="failed to get container status \"d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d\": rpc error: code = NotFound desc = could not find container \"d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d\": container with ID starting with d1412688091756f315728b6f3321207b6ad6170b3c22add79b4a9c0dec0b7c4d not found: ID does not exist"
Sep 07 11:25:30.172802 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172771    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6mfn\" (UniqueName: \"kubernetes.io/projected/8f6469c2-c37f-4114-b026-40c2e1c35860-kube-api-access-w6mfn\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.172919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172806    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-oauth-config\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.172919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172820    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-trusted-ca-bundle\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.172919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172841    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.172919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172855    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-oauth-serving-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.172919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172869    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f6469c2-c37f-4114-b026-40c2e1c35860-console-serving-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.172919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.172882    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f6469c2-c37f-4114-b026-40c2e1c35860-console-config\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:30.176273 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.174490    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6658b8986f-k24qn"]
Sep 07 11:25:30.178624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.178603    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6658b8986f-k24qn"]
Sep 07 11:25:30.496697 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:30.496667    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f6469c2-c37f-4114-b026-40c2e1c35860" path="/var/lib/kubelet/pods/8f6469c2-c37f-4114-b026-40c2e1c35860/volumes"
Sep 07 11:25:40.172432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.172384    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9688ccf8b-tzk5b" podUID="b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" containerName="console" containerID="cri-o://df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65" gracePeriod=15
Sep 07 11:25:40.446129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.446065    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:25:40.557523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.557496    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-oauth-config\") pod \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") "
Sep 07 11:25:40.557685 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.557555    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-oauth-serving-cert\") pod \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") "
Sep 07 11:25:40.557685 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.557594    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-serving-cert\") pod \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") "
Sep 07 11:25:40.557685 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.557614    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-service-ca\") pod \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") "
Sep 07 11:25:40.557685 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.557632    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sdpq\" (UniqueName: \"kubernetes.io/projected/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-kube-api-access-8sdpq\") pod \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") "
Sep 07 11:25:40.557918 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.557724    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-config\") pod \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\" (UID: \"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5\") "
Sep 07 11:25:40.558099 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.558060    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" (UID: "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:40.558164 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.558066    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-service-ca" (OuterVolumeSpecName: "service-ca") pod "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" (UID: "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:40.558164 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.558114    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-config" (OuterVolumeSpecName: "console-config") pod "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" (UID: "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:25:40.559769 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.559725    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" (UID: "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:25:40.559769 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.559759    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" (UID: "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:25:40.559978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.559958    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-kube-api-access-8sdpq" (OuterVolumeSpecName: "kube-api-access-8sdpq") pod "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" (UID: "b98d02bb-f2c9-4e73-8ac4-1de26f0613d5"). InnerVolumeSpecName "kube-api-access-8sdpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:25:40.658732 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.658698    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-config\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:40.658732 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.658726    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-oauth-config\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:40.658732 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.658764    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-oauth-serving-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:40.658978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.658777    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-console-serving-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:40.658978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.658792    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:40.658978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:40.658802    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sdpq\" (UniqueName: \"kubernetes.io/projected/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5-kube-api-access-8sdpq\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:25:41.196385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.196303    2560 generic.go:356] "Generic (PLEG): container finished" podID="b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" containerID="df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65" exitCode=0
Sep 07 11:25:41.196385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.196352    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9688ccf8b-tzk5b"
Sep 07 11:25:41.196834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.196398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9688ccf8b-tzk5b" event={"ID":"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5","Type":"ContainerDied","Data":"df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65"}
Sep 07 11:25:41.196834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.196452    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9688ccf8b-tzk5b" event={"ID":"b98d02bb-f2c9-4e73-8ac4-1de26f0613d5","Type":"ContainerDied","Data":"6486950ddb0eda910d7528ad236666785536a54c209ea562a046792ab1b78d60"}
Sep 07 11:25:41.196834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.196476    2560 scope.go:117] "RemoveContainer" containerID="df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65"
Sep 07 11:25:41.205079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.205063    2560 scope.go:117] "RemoveContainer" containerID="df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65"
Sep 07 11:25:41.205368 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:25:41.205344    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65\": container with ID starting with df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65 not found: ID does not exist" containerID="df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65"
Sep 07 11:25:41.205460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.205373    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65"} err="failed to get container status \"df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65\": rpc error: code = NotFound desc = could not find container \"df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65\": container with ID starting with df111e91cb17df0153315346daf294af37433dbd69b18f70e1085ca9af218b65 not found: ID does not exist"
Sep 07 11:25:41.218656 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.218630    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9688ccf8b-tzk5b"]
Sep 07 11:25:41.222626 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:41.222607    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9688ccf8b-tzk5b"]
Sep 07 11:25:42.496321 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:42.496288    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" path="/var/lib/kubelet/pods/b98d02bb-f2c9-4e73-8ac4-1de26f0613d5/volumes"
Sep 07 11:25:59.511324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:59.511280    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:25:59.531096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:25:59.531064    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:26:00.285802 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:00.285770    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:26:26.980346 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:26.980302    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76bbf9b6d5-fhvhj"]
Sep 07 11:26:52.000388 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.000323    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76bbf9b6d5-fhvhj" podUID="84050e73-55c7-4cad-9752-c13a36c4c6b5" containerName="console" containerID="cri-o://85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a" gracePeriod=15
Sep 07 11:26:52.242225 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.242198    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:26:52.333135 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333040    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-serving-cert\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333135 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333081    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-service-ca\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333135 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333107    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-oauth-serving-cert\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-oauth-config\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333167    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw4rr\" (UniqueName: \"kubernetes.io/projected/84050e73-55c7-4cad-9752-c13a36c4c6b5-kube-api-access-dw4rr\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333181    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-config\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333210    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-trusted-ca-bundle\") pod \"84050e73-55c7-4cad-9752-c13a36c4c6b5\" (UID: \"84050e73-55c7-4cad-9752-c13a36c4c6b5\") "
Sep 07 11:26:52.333645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333594    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-service-ca" (OuterVolumeSpecName: "service-ca") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:26:52.333697 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333665    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:26:52.333773 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333689    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-config" (OuterVolumeSpecName: "console-config") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:26:52.333773 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.333717    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:26:52.335285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.335262    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:26:52.335812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.335784    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:26:52.335912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.335825    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84050e73-55c7-4cad-9752-c13a36c4c6b5-kube-api-access-dw4rr" (OuterVolumeSpecName: "kube-api-access-dw4rr") pod "84050e73-55c7-4cad-9752-c13a36c4c6b5" (UID: "84050e73-55c7-4cad-9752-c13a36c4c6b5"). InnerVolumeSpecName "kube-api-access-dw4rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:26:52.433963 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.433922    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.433963 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.433958    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-oauth-serving-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.433963 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.433969    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-oauth-config\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.434197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.433978    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw4rr\" (UniqueName: \"kubernetes.io/projected/84050e73-55c7-4cad-9752-c13a36c4c6b5-kube-api-access-dw4rr\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.434197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.433988    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-config\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.434197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.433996    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84050e73-55c7-4cad-9752-c13a36c4c6b5-trusted-ca-bundle\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.434197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.434004    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84050e73-55c7-4cad-9752-c13a36c4c6b5-console-serving-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:26:52.455543 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.455509    2560 generic.go:356] "Generic (PLEG): container finished" podID="84050e73-55c7-4cad-9752-c13a36c4c6b5" containerID="85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a" exitCode=0
Sep 07 11:26:52.455684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.455553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bbf9b6d5-fhvhj" event={"ID":"84050e73-55c7-4cad-9752-c13a36c4c6b5","Type":"ContainerDied","Data":"85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a"}
Sep 07 11:26:52.455684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.455563    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bbf9b6d5-fhvhj"
Sep 07 11:26:52.455684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.455605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bbf9b6d5-fhvhj" event={"ID":"84050e73-55c7-4cad-9752-c13a36c4c6b5","Type":"ContainerDied","Data":"63d406c52119e6e7eac42ea8875e66346586354d7391fa8bbbfb65217298de5c"}
Sep 07 11:26:52.455684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.455633    2560 scope.go:117] "RemoveContainer" containerID="85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a"
Sep 07 11:26:52.463828 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.463808    2560 scope.go:117] "RemoveContainer" containerID="85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a"
Sep 07 11:26:52.464116 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:26:52.464097    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a\": container with ID starting with 85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a not found: ID does not exist" containerID="85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a"
Sep 07 11:26:52.464171 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.464125    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a"} err="failed to get container status \"85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a\": rpc error: code = NotFound desc = could not find container \"85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a\": container with ID starting with 85db10012441b66aa0184e190ba965ec438064ca3742d61681c4ee2f93948b9a not found: ID does not exist"
Sep 07 11:26:52.474712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.474688    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76bbf9b6d5-fhvhj"]
Sep 07 11:26:52.480410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.480388    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76bbf9b6d5-fhvhj"]
Sep 07 11:26:52.495730 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:26:52.495708    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84050e73-55c7-4cad-9752-c13a36c4c6b5" path="/var/lib/kubelet/pods/84050e73-55c7-4cad-9752-c13a36c4c6b5/volumes"
Sep 07 11:27:48.452189 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:27:48.452156    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:27:48.452723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:27:48.452318    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:28:09.148420 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148384    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"]
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148640    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84050e73-55c7-4cad-9752-c13a36c4c6b5" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148650    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="84050e73-55c7-4cad-9752-c13a36c4c6b5" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148660    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f6469c2-c37f-4114-b026-40c2e1c35860" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148666    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f6469c2-c37f-4114-b026-40c2e1c35860" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148674    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148680    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148735    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f6469c2-c37f-4114-b026-40c2e1c35860" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148759    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98d02bb-f2c9-4e73-8ac4-1de26f0613d5" containerName="console"
Sep 07 11:28:09.150754 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.148766    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="84050e73-55c7-4cad-9752-c13a36c4c6b5" containerName="console"
Sep 07 11:28:09.151620 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.151604    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.163695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.160008    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 07 11:28:09.163695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.160292    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 07 11:28:09.163695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.162877    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"]
Sep 07 11:28:09.163695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.163042    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2pzfr\""
Sep 07 11:28:09.202267 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.201133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b9caa79-2e2a-43b9-b10f-5091fd669460-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4hs7t\" (UID: \"0b9caa79-2e2a-43b9-b10f-5091fd669460\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.202267 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.201220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qqjm\" (UniqueName: \"kubernetes.io/projected/0b9caa79-2e2a-43b9-b10f-5091fd669460-kube-api-access-6qqjm\") pod \"cert-manager-cainjector-559cdd8c87-4hs7t\" (UID: \"0b9caa79-2e2a-43b9-b10f-5091fd669460\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.302560 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.302519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b9caa79-2e2a-43b9-b10f-5091fd669460-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4hs7t\" (UID: \"0b9caa79-2e2a-43b9-b10f-5091fd669460\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.302726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.302595    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qqjm\" (UniqueName: \"kubernetes.io/projected/0b9caa79-2e2a-43b9-b10f-5091fd669460-kube-api-access-6qqjm\") pod \"cert-manager-cainjector-559cdd8c87-4hs7t\" (UID: \"0b9caa79-2e2a-43b9-b10f-5091fd669460\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.310621 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.310589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b9caa79-2e2a-43b9-b10f-5091fd669460-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4hs7t\" (UID: \"0b9caa79-2e2a-43b9-b10f-5091fd669460\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.310826 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.310804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qqjm\" (UniqueName: \"kubernetes.io/projected/0b9caa79-2e2a-43b9-b10f-5091fd669460-kube-api-access-6qqjm\") pod \"cert-manager-cainjector-559cdd8c87-4hs7t\" (UID: \"0b9caa79-2e2a-43b9-b10f-5091fd669460\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.486263 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.486229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"
Sep 07 11:28:09.755394 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.755369    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t"]
Sep 07 11:28:09.758130 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:28:09.758102    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b9caa79_2e2a_43b9_b10f_5091fd669460.slice/crio-35b320704d5ef65142e90dfdb411ee0dda5b56afbbd33ec6452b4d3fc7d1962d WatchSource:0}: Error finding container 35b320704d5ef65142e90dfdb411ee0dda5b56afbbd33ec6452b4d3fc7d1962d: Status 404 returned error can't find the container with id 35b320704d5ef65142e90dfdb411ee0dda5b56afbbd33ec6452b4d3fc7d1962d
Sep 07 11:28:09.760499 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:09.760483    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:28:10.738844 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:10.738805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t" event={"ID":"0b9caa79-2e2a-43b9-b10f-5091fd669460","Type":"ContainerStarted","Data":"35b320704d5ef65142e90dfdb411ee0dda5b56afbbd33ec6452b4d3fc7d1962d"}
Sep 07 11:28:15.759535 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:15.759500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t" event={"ID":"0b9caa79-2e2a-43b9-b10f-5091fd669460","Type":"ContainerStarted","Data":"c94029a45c17263950bc97e5cba8b69a4b422e37dede90c4238aefdc2de827f6"}
Sep 07 11:28:16.785340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:16.785283    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4hs7t" podStartSLOduration=1.844050016 podStartE2EDuration="7.785267741s" totalImagesPullingTime="5.941217725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:28:16.78338943 +0000 UTC m=+328.730740214" watchObservedRunningTime="2026-09-07 11:28:16.785267741 +0000 UTC m=+328.732618524"
Sep 07 11:28:43.141043 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.140968    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"]
Sep 07 11:28:43.144034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.144017    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.146770 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.146728    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 07 11:28:43.146888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.146804    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 07 11:28:43.146888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.146849    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8z5pf\""
Sep 07 11:28:43.146990 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.146854    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 07 11:28:43.146990 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.146854    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 07 11:28:43.147063 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.146995    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:28:43.158265 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.158243    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"]
Sep 07 11:28:43.263415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.263347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1046905b-f697-422f-96ad-75be2d3a9ff8-cert\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.263603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.263436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1046905b-f697-422f-96ad-75be2d3a9ff8-metrics-cert\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.263603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.263458    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1046905b-f697-422f-96ad-75be2d3a9ff8-manager-config\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.263603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.263496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxlfd\" (UniqueName: \"kubernetes.io/projected/1046905b-f697-422f-96ad-75be2d3a9ff8-kube-api-access-wxlfd\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.364839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.364796    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxlfd\" (UniqueName: \"kubernetes.io/projected/1046905b-f697-422f-96ad-75be2d3a9ff8-kube-api-access-wxlfd\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.365041 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.364857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1046905b-f697-422f-96ad-75be2d3a9ff8-cert\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.365041 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.364914    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1046905b-f697-422f-96ad-75be2d3a9ff8-metrics-cert\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.365041 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.364942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1046905b-f697-422f-96ad-75be2d3a9ff8-manager-config\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.365677 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.365650    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1046905b-f697-422f-96ad-75be2d3a9ff8-manager-config\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.367388 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.367308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1046905b-f697-422f-96ad-75be2d3a9ff8-cert\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.367388 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.367345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1046905b-f697-422f-96ad-75be2d3a9ff8-metrics-cert\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.375924 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.375902    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxlfd\" (UniqueName: \"kubernetes.io/projected/1046905b-f697-422f-96ad-75be2d3a9ff8-kube-api-access-wxlfd\") pod \"lws-controller-manager-79b5d8b54d-s6pz6\" (UID: \"1046905b-f697-422f-96ad-75be2d3a9ff8\") " pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.453732 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.453642    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:43.732763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.732709    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"]
Sep 07 11:28:43.736142 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:28:43.736111    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1046905b_f697_422f_96ad_75be2d3a9ff8.slice/crio-bcfad46ecdf3e97163fd3e91193f44e417e06a5472377c70670a6306131c9a14 WatchSource:0}: Error finding container bcfad46ecdf3e97163fd3e91193f44e417e06a5472377c70670a6306131c9a14: Status 404 returned error can't find the container with id bcfad46ecdf3e97163fd3e91193f44e417e06a5472377c70670a6306131c9a14
Sep 07 11:28:43.858851 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:43.858811    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6" event={"ID":"1046905b-f697-422f-96ad-75be2d3a9ff8","Type":"ContainerStarted","Data":"bcfad46ecdf3e97163fd3e91193f44e417e06a5472377c70670a6306131c9a14"}
Sep 07 11:28:45.757948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.757912    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"]
Sep 07 11:28:45.776199 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.776173    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.778980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.778841    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 11:28:45.778980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.778863    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 11:28:45.778980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.778955    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 11:28:45.779236 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.779216    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 11:28:45.779295 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.779237    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g8bwf\""
Sep 07 11:28:45.789073 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.789051    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"]
Sep 07 11:28:45.887478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.887191    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00c310e0-8385-4e86-9bb2-96e92a5ab804-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.887478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.887236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb24z\" (UniqueName: \"kubernetes.io/projected/00c310e0-8385-4e86-9bb2-96e92a5ab804-kube-api-access-tb24z\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.887478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.887301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00c310e0-8385-4e86-9bb2-96e92a5ab804-manifests\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.887478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.887365    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00c310e0-8385-4e86-9bb2-96e92a5ab804-webhook-cert\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.887478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.887386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c310e0-8385-4e86-9bb2-96e92a5ab804-tmp\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.988054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.988020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00c310e0-8385-4e86-9bb2-96e92a5ab804-manifests\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.988230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.988081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00c310e0-8385-4e86-9bb2-96e92a5ab804-webhook-cert\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.988230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.988109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c310e0-8385-4e86-9bb2-96e92a5ab804-tmp\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.988230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.988156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00c310e0-8385-4e86-9bb2-96e92a5ab804-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.988230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.988185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb24z\" (UniqueName: \"kubernetes.io/projected/00c310e0-8385-4e86-9bb2-96e92a5ab804-kube-api-access-tb24z\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.988568 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.988541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/00c310e0-8385-4e86-9bb2-96e92a5ab804-manifests\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.990758 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.990713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c310e0-8385-4e86-9bb2-96e92a5ab804-tmp\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.991084 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.991062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/00c310e0-8385-4e86-9bb2-96e92a5ab804-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.991180 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.991107    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00c310e0-8385-4e86-9bb2-96e92a5ab804-webhook-cert\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:45.996298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:45.996272    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb24z\" (UniqueName: \"kubernetes.io/projected/00c310e0-8385-4e86-9bb2-96e92a5ab804-kube-api-access-tb24z\") pod \"opendatahub-operator-controller-manager-f479d7474-pvz66\" (UID: \"00c310e0-8385-4e86-9bb2-96e92a5ab804\") " pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:46.089186 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:46.089103    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:46.379391 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:46.379311    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"]
Sep 07 11:28:46.382294 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:28:46.382267    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c310e0_8385_4e86_9bb2_96e92a5ab804.slice/crio-0886c71afd0fd851ebb747d32541ad5349a4511b3e568ab3983a01a2a4a4b44d WatchSource:0}: Error finding container 0886c71afd0fd851ebb747d32541ad5349a4511b3e568ab3983a01a2a4a4b44d: Status 404 returned error can't find the container with id 0886c71afd0fd851ebb747d32541ad5349a4511b3e568ab3983a01a2a4a4b44d
Sep 07 11:28:46.872097 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:46.872062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66" event={"ID":"00c310e0-8385-4e86-9bb2-96e92a5ab804","Type":"ContainerStarted","Data":"0886c71afd0fd851ebb747d32541ad5349a4511b3e568ab3983a01a2a4a4b44d"}
Sep 07 11:28:46.873502 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:46.873471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6" event={"ID":"1046905b-f697-422f-96ad-75be2d3a9ff8","Type":"ContainerStarted","Data":"4ffa86de1b41f08c178bbc2989fe4a7518b45b9383ab2e66f27dbc680cea0fe1"}
Sep 07 11:28:46.873632 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:46.873598    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:28:46.889222 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:46.889177    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6" podStartSLOduration=1.403429088 podStartE2EDuration="3.889161569s" totalImagesPullingTime="2.485732481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:28:46.888588449 +0000 UTC m=+358.835939232" watchObservedRunningTime="2026-09-07 11:28:46.889161569 +0000 UTC m=+358.836512353"
Sep 07 11:28:49.887379 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:49.887341    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66" event={"ID":"00c310e0-8385-4e86-9bb2-96e92a5ab804","Type":"ContainerStarted","Data":"47b0542e5f37b92c1578c7187ef222ce538e964a9561edd961887360cfc63f8a"}
Sep 07 11:28:50.893157 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:50.893065    2560 generic.go:356] "Generic (PLEG): container finished" podID="00c310e0-8385-4e86-9bb2-96e92a5ab804" containerID="47b0542e5f37b92c1578c7187ef222ce538e964a9561edd961887360cfc63f8a" exitCode=0
Sep 07 11:28:50.893157 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:50.893146    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66" event={"ID":"00c310e0-8385-4e86-9bb2-96e92a5ab804","Type":"ContainerDied","Data":"47b0542e5f37b92c1578c7187ef222ce538e964a9561edd961887360cfc63f8a"}
Sep 07 11:28:51.900255 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:51.900204    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66" event={"ID":"00c310e0-8385-4e86-9bb2-96e92a5ab804","Type":"ContainerStarted","Data":"4b3ead8b591917dfaf7c8ce15fb42e5ec567d79c5b1fa7a216f227f00e7d64c5"}
Sep 07 11:28:51.900771 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:51.900317    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:28:51.924216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:51.924152    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66" podStartSLOduration=2.438572879 podStartE2EDuration="6.924133288s" totalImagesPullingTime="3.446532652s" totalInitContainerRuntime="1.039027757s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:28:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:28:51.922358902 +0000 UTC m=+363.869709687" watchObservedRunningTime="2026-09-07 11:28:51.924133288 +0000 UTC m=+363.871484072"
Sep 07 11:28:57.880358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:28:57.880327    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-79b5d8b54d-s6pz6"
Sep 07 11:29:02.907442 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:02.907413    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f479d7474-pvz66"
Sep 07 11:29:13.814541 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.814505    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"]
Sep 07 11:29:13.817618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.817597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.820061 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.820030    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-77zkl\""
Sep 07 11:29:13.820200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.820063    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 11:29:13.820200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.820087    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 11:29:13.820400 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.820378    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 11:29:13.828369 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.828347    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"]
Sep 07 11:29:13.915946 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.915908    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.915946 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.915957    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7642862-5cb6-44a0-9100-311f0a5f9156-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.915992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.916041    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.916066    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.916086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.916110    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916300 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.916148    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ts2\" (UniqueName: \"kubernetes.io/projected/c7642862-5cb6-44a0-9100-311f0a5f9156-kube-api-access-j6ts2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:13.916300 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:13.916200    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016560 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016524    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016560 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016566    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016593    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016650    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ts2\" (UniqueName: \"kubernetes.io/projected/c7642862-5cb6-44a0-9100-311f0a5f9156-kube-api-access-j6ts2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016781    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7642862-5cb6-44a0-9100-311f0a5f9156-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.016920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.016814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.017335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.017265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.017335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.017313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.017476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.017455    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.017519 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.017474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.017820 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.017802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7642862-5cb6-44a0-9100-311f0a5f9156-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.019056 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.019038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.019386 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.019370    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.023815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.023792    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7642862-5cb6-44a0-9100-311f0a5f9156-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.023962 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.023947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ts2\" (UniqueName: \"kubernetes.io/projected/c7642862-5cb6-44a0-9100-311f0a5f9156-kube-api-access-j6ts2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch\" (UID: \"c7642862-5cb6-44a0-9100-311f0a5f9156\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.129166 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.129076    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:14.403098 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.403070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"]
Sep 07 11:29:14.405460 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:29:14.405428    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7642862_5cb6_44a0_9100_311f0a5f9156.slice/crio-20216d5f3ff922bb4af1f02c49e1bd091fbec34cfc27b9508e03cabd18c2edec WatchSource:0}: Error finding container 20216d5f3ff922bb4af1f02c49e1bd091fbec34cfc27b9508e03cabd18c2edec: Status 404 returned error can't find the container with id 20216d5f3ff922bb4af1f02c49e1bd091fbec34cfc27b9508e03cabd18c2edec
Sep 07 11:29:14.986547 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:14.986509    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch" event={"ID":"c7642862-5cb6-44a0-9100-311f0a5f9156","Type":"ContainerStarted","Data":"20216d5f3ff922bb4af1f02c49e1bd091fbec34cfc27b9508e03cabd18c2edec"}
Sep 07 11:29:17.028482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.028440    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:29:17.028765 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.028517    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:29:17.028765 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.028557    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:29:17.283806 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.283699    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h9tlh"]
Sep 07 11:29:17.292993 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.292962    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.295598 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.295219    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 07 11:29:17.295598 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.295452    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pg5th\""
Sep 07 11:29:17.298763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.298675    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h9tlh"]
Sep 07 11:29:17.343154 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.342917    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp8b4\" (UniqueName: \"kubernetes.io/projected/2550995d-e8a3-4c74-9b2e-345f9ce98640-kube-api-access-sp8b4\") pod \"odh-model-controller-7998c67dc4-h9tlh\" (UID: \"2550995d-e8a3-4c74-9b2e-345f9ce98640\") " pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.343154 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.343037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2550995d-e8a3-4c74-9b2e-345f9ce98640-cert\") pod \"odh-model-controller-7998c67dc4-h9tlh\" (UID: \"2550995d-e8a3-4c74-9b2e-345f9ce98640\") " pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.443434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.443395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp8b4\" (UniqueName: \"kubernetes.io/projected/2550995d-e8a3-4c74-9b2e-345f9ce98640-kube-api-access-sp8b4\") pod \"odh-model-controller-7998c67dc4-h9tlh\" (UID: \"2550995d-e8a3-4c74-9b2e-345f9ce98640\") " pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.443646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.443489    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2550995d-e8a3-4c74-9b2e-345f9ce98640-cert\") pod \"odh-model-controller-7998c67dc4-h9tlh\" (UID: \"2550995d-e8a3-4c74-9b2e-345f9ce98640\") " pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.446395 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.446365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2550995d-e8a3-4c74-9b2e-345f9ce98640-cert\") pod \"odh-model-controller-7998c67dc4-h9tlh\" (UID: \"2550995d-e8a3-4c74-9b2e-345f9ce98640\") " pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.452485 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.452428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp8b4\" (UniqueName: \"kubernetes.io/projected/2550995d-e8a3-4c74-9b2e-345f9ce98640-kube-api-access-sp8b4\") pod \"odh-model-controller-7998c67dc4-h9tlh\" (UID: \"2550995d-e8a3-4c74-9b2e-345f9ce98640\") " pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.612194 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.612089    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:17.731006 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.730972    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-89plz"]
Sep 07 11:29:17.736557 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.736532    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:17.738975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.738802    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 07 11:29:17.744450 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.744417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-89plz"]
Sep 07 11:29:17.746469 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.746444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwgn7\" (UniqueName: \"kubernetes.io/projected/3118cf26-0bbc-4feb-96da-ba41f9a00b00-kube-api-access-dwgn7\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:17.746586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.746540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3118cf26-0bbc-4feb-96da-ba41f9a00b00-tls-certs\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:17.847600 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.847547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwgn7\" (UniqueName: \"kubernetes.io/projected/3118cf26-0bbc-4feb-96da-ba41f9a00b00-kube-api-access-dwgn7\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:17.847811 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.847656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3118cf26-0bbc-4feb-96da-ba41f9a00b00-tls-certs\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:17.847923 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:29:17.847850    2560 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 07 11:29:17.847988 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:29:17.847933    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3118cf26-0bbc-4feb-96da-ba41f9a00b00-tls-certs podName:3118cf26-0bbc-4feb-96da-ba41f9a00b00 nodeName:}" failed. No retries permitted until 2026-09-07 11:29:18.347912924 +0000 UTC m=+390.295263710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3118cf26-0bbc-4feb-96da-ba41f9a00b00-tls-certs") pod "model-serving-api-54d89bd5fc-89plz" (UID: "3118cf26-0bbc-4feb-96da-ba41f9a00b00") : secret "model-serving-api-tls" not found
Sep 07 11:29:17.857336 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.857301    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwgn7\" (UniqueName: \"kubernetes.io/projected/3118cf26-0bbc-4feb-96da-ba41f9a00b00-kube-api-access-dwgn7\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:17.898794 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:17.898764    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h9tlh"]
Sep 07 11:29:17.901035 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:29:17.901008    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2550995d_e8a3_4c74_9b2e_345f9ce98640.slice/crio-93c0458a78a60384a0598a8b9defe2cbb294e411b6920a96b6242be74d3e29ff WatchSource:0}: Error finding container 93c0458a78a60384a0598a8b9defe2cbb294e411b6920a96b6242be74d3e29ff: Status 404 returned error can't find the container with id 93c0458a78a60384a0598a8b9defe2cbb294e411b6920a96b6242be74d3e29ff
Sep 07 11:29:18.003400 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.003354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch" event={"ID":"c7642862-5cb6-44a0-9100-311f0a5f9156","Type":"ContainerStarted","Data":"d2ae898bb47a1bc472388fb9eb9aafa661287def25817f56122c0216668cee23"}
Sep 07 11:29:18.004789 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.004760    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh" event={"ID":"2550995d-e8a3-4c74-9b2e-345f9ce98640","Type":"ContainerStarted","Data":"93c0458a78a60384a0598a8b9defe2cbb294e411b6920a96b6242be74d3e29ff"}
Sep 07 11:29:18.024542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.024476    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch" podStartSLOduration=2.404217652 podStartE2EDuration="5.024459877s" totalImagesPullingTime="2.620242225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:29:18.023444274 +0000 UTC m=+389.970795060" watchObservedRunningTime="2026-09-07 11:29:18.024459877 +0000 UTC m=+389.971810661"
Sep 07 11:29:18.129428 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.129392    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:18.134389 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.134320    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:18.353785 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.353718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3118cf26-0bbc-4feb-96da-ba41f9a00b00-tls-certs\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:18.356481 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.356449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3118cf26-0bbc-4feb-96da-ba41f9a00b00-tls-certs\") pod \"model-serving-api-54d89bd5fc-89plz\" (UID: \"3118cf26-0bbc-4feb-96da-ba41f9a00b00\") " pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:18.650117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.649972    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:18.943665 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:18.943633    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-89plz"]
Sep 07 11:29:18.946884 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:29:18.946850    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3118cf26_0bbc_4feb_96da_ba41f9a00b00.slice/crio-b3e83ac98f573d955c98091f2a304dbd5a2390109831d2afe22bcbc53a760133 WatchSource:0}: Error finding container b3e83ac98f573d955c98091f2a304dbd5a2390109831d2afe22bcbc53a760133: Status 404 returned error can't find the container with id b3e83ac98f573d955c98091f2a304dbd5a2390109831d2afe22bcbc53a760133
Sep 07 11:29:19.011126 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:19.011067    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-89plz" event={"ID":"3118cf26-0bbc-4feb-96da-ba41f9a00b00","Type":"ContainerStarted","Data":"b3e83ac98f573d955c98091f2a304dbd5a2390109831d2afe22bcbc53a760133"}
Sep 07 11:29:19.011363 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:19.011341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:19.012855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:19.012831    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch"
Sep 07 11:29:23.034081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:23.034039    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh" event={"ID":"2550995d-e8a3-4c74-9b2e-345f9ce98640","Type":"ContainerStarted","Data":"c0ebf9ff2729a8cd8a74fd92f8dfa204f6f14c24ed3cf760d03902361b44e7b3"}
Sep 07 11:29:23.034558 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:23.034230    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:23.035966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:23.035919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-89plz" event={"ID":"3118cf26-0bbc-4feb-96da-ba41f9a00b00","Type":"ContainerStarted","Data":"868f9cf687970fd372a60ee640862a48fc99de80aa253d9cb5cc62943c815911"}
Sep 07 11:29:23.036084 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:23.036005    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:23.050277 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:23.050218    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh" podStartSLOduration=1.6817892730000001 podStartE2EDuration="6.05020443s" totalImagesPullingTime="4.368415157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:29:23.048805049 +0000 UTC m=+394.996155834" watchObservedRunningTime="2026-09-07 11:29:23.05020443 +0000 UTC m=+394.997555213"
Sep 07 11:29:23.064389 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:23.064345    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-89plz" podStartSLOduration=2.7036808839999997 podStartE2EDuration="6.064321672s" totalImagesPullingTime="3.360640788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:29:23.062624235 +0000 UTC m=+395.009975019" watchObservedRunningTime="2026-09-07 11:29:23.064321672 +0000 UTC m=+395.011672457"
Sep 07 11:29:34.042581 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:34.042551    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-h9tlh"
Sep 07 11:29:34.044788 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:34.044767    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-89plz"
Sep 07 11:29:40.765829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.765792    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77886d4955-865xr"]
Sep 07 11:29:40.773191 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.773165    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.775190 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.775165    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 07 11:29:40.775458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.775439    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 07 11:29:40.775863 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.775847    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 07 11:29:40.776152 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.776132    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 07 11:29:40.776294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.776171    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 07 11:29:40.776384 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.776361    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 07 11:29:40.776534 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.776504    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k84wv\""
Sep 07 11:29:40.776596 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.776566    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 07 11:29:40.779521 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.779500    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77886d4955-865xr"]
Sep 07 11:29:40.781609 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.781590    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 07 11:29:40.856991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.856954    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-service-ca\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.856991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.856994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-trusted-ca-bundle\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.857224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.857022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8gz4\" (UniqueName: \"kubernetes.io/projected/5bf0a0e9-d068-44c9-a859-a86ae59865b9-kube-api-access-n8gz4\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.857224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.857129    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-config\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.857224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.857159    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-oauth-serving-cert\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.857224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.857186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-serving-cert\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.857400 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.857238    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-oauth-config\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gz4\" (UniqueName: \"kubernetes.io/projected/5bf0a0e9-d068-44c9-a859-a86ae59865b9-kube-api-access-n8gz4\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958267 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-config\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958267 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-oauth-serving-cert\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958267 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958228    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-serving-cert\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958428 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-oauth-config\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958428 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-service-ca\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958428 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958373    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-trusted-ca-bundle\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.958895 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958873    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-oauth-serving-cert\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.959017 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.958947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-config\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.959151 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.959125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-service-ca\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.959214 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.959165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf0a0e9-d068-44c9-a859-a86ae59865b9-trusted-ca-bundle\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.960712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.960678    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-serving-cert\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.960712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.960690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bf0a0e9-d068-44c9-a859-a86ae59865b9-console-oauth-config\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:40.965822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:40.965803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gz4\" (UniqueName: \"kubernetes.io/projected/5bf0a0e9-d068-44c9-a859-a86ae59865b9-kube-api-access-n8gz4\") pod \"console-77886d4955-865xr\" (UID: \"5bf0a0e9-d068-44c9-a859-a86ae59865b9\") " pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:41.083672 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:41.083589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:41.360340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:41.360249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77886d4955-865xr"]
Sep 07 11:29:41.363081 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:29:41.363051    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf0a0e9_d068_44c9_a859_a86ae59865b9.slice/crio-cd56743e5587664874613b3f6c856b96311510bba3e580eef5b4f5a5f2f99b3e WatchSource:0}: Error finding container cd56743e5587664874613b3f6c856b96311510bba3e580eef5b4f5a5f2f99b3e: Status 404 returned error can't find the container with id cd56743e5587664874613b3f6c856b96311510bba3e580eef5b4f5a5f2f99b3e
Sep 07 11:29:42.120172 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:42.120134    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77886d4955-865xr" event={"ID":"5bf0a0e9-d068-44c9-a859-a86ae59865b9","Type":"ContainerStarted","Data":"46ff51363b768e1e21ea5a483b51580ec0b50a6f5d178f26f9e9a97aa3f9176d"}
Sep 07 11:29:42.120556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:42.120189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77886d4955-865xr" event={"ID":"5bf0a0e9-d068-44c9-a859-a86ae59865b9","Type":"ContainerStarted","Data":"cd56743e5587664874613b3f6c856b96311510bba3e580eef5b4f5a5f2f99b3e"}
Sep 07 11:29:42.137761 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:42.137692    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77886d4955-865xr" podStartSLOduration=2.13767512 podStartE2EDuration="2.13767512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:29:42.137041981 +0000 UTC m=+414.084392767" watchObservedRunningTime="2026-09-07 11:29:42.13767512 +0000 UTC m=+414.085025903"
Sep 07 11:29:51.084821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:51.084783    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:51.085228 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:51.084843    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:51.089725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:51.089706    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:29:51.159964 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:29:51.159932    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77886d4955-865xr"
Sep 07 11:30:46.673551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.673511    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"]
Sep 07 11:30:46.677119 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.677102    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.679422 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.679391    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6brmr\""
Sep 07 11:30:46.679791 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.679774    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 11:30:46.680153 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.680133    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 11:30:46.690648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.690621    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"]
Sep 07 11:30:46.716224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.716112    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njqc\" (UniqueName: \"kubernetes.io/projected/b34e7048-afe4-484f-b90a-65e11ca28206-kube-api-access-8njqc\") pod \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.716224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.716177    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34e7048-afe4-484f-b90a-65e11ca28206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.816893 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.816863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8njqc\" (UniqueName: \"kubernetes.io/projected/b34e7048-afe4-484f-b90a-65e11ca28206-kube-api-access-8njqc\") pod \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.817081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.816916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34e7048-afe4-484f-b90a-65e11ca28206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.817294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.817274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34e7048-afe4-484f-b90a-65e11ca28206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.835796 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.835763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njqc\" (UniqueName: \"kubernetes.io/projected/b34e7048-afe4-484f-b90a-65e11ca28206-kube-api-access-8njqc\") pod \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:46.988357 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:46.988331    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:47.272413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:47.272327    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"]
Sep 07 11:30:47.275104 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:30:47.275062    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34e7048_afe4_484f_b90a_65e11ca28206.slice/crio-02a9ded80769c1051f63ced3c6ff9f4932e5d0d1c9159edaab5a90debec76fcc WatchSource:0}: Error finding container 02a9ded80769c1051f63ced3c6ff9f4932e5d0d1c9159edaab5a90debec76fcc: Status 404 returned error can't find the container with id 02a9ded80769c1051f63ced3c6ff9f4932e5d0d1c9159edaab5a90debec76fcc
Sep 07 11:30:47.406503 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:47.406472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw" event={"ID":"b34e7048-afe4-484f-b90a-65e11ca28206","Type":"ContainerStarted","Data":"02a9ded80769c1051f63ced3c6ff9f4932e5d0d1c9159edaab5a90debec76fcc"}
Sep 07 11:30:51.537497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.537466    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"]
Sep 07 11:30:51.540070 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.540051    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:30:51.542131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.542107    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 07 11:30:51.542557 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.542539    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zhtfw\""
Sep 07 11:30:51.550943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.550922    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"]
Sep 07 11:30:51.551974 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.551712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v4qs\" (UniqueName: \"kubernetes.io/projected/edf3f957-8a75-445e-8780-be269da59b07-kube-api-access-4v4qs\") pod \"dns-operator-controller-manager-5985564776-4xmxz\" (UID: \"edf3f957-8a75-445e-8780-be269da59b07\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:30:51.652609 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.652573    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v4qs\" (UniqueName: \"kubernetes.io/projected/edf3f957-8a75-445e-8780-be269da59b07-kube-api-access-4v4qs\") pod \"dns-operator-controller-manager-5985564776-4xmxz\" (UID: \"edf3f957-8a75-445e-8780-be269da59b07\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:30:51.661807 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.661773    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v4qs\" (UniqueName: \"kubernetes.io/projected/edf3f957-8a75-445e-8780-be269da59b07-kube-api-access-4v4qs\") pod \"dns-operator-controller-manager-5985564776-4xmxz\" (UID: \"edf3f957-8a75-445e-8780-be269da59b07\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:30:51.850196 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:51.850143    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:30:52.133048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:52.132966    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"]
Sep 07 11:30:52.135996 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:30:52.135970    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf3f957_8a75_445e_8780_be269da59b07.slice/crio-de3634ca316f64a64f293973d2d58d5a7aa912d5cfa8fb3de9300e5e95f619c4 WatchSource:0}: Error finding container de3634ca316f64a64f293973d2d58d5a7aa912d5cfa8fb3de9300e5e95f619c4: Status 404 returned error can't find the container with id de3634ca316f64a64f293973d2d58d5a7aa912d5cfa8fb3de9300e5e95f619c4
Sep 07 11:30:52.434515 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:52.434427    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw" event={"ID":"b34e7048-afe4-484f-b90a-65e11ca28206","Type":"ContainerStarted","Data":"3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad"}
Sep 07 11:30:52.434672 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:52.434513    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:30:52.435837 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:52.435812    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz" event={"ID":"edf3f957-8a75-445e-8780-be269da59b07","Type":"ContainerStarted","Data":"de3634ca316f64a64f293973d2d58d5a7aa912d5cfa8fb3de9300e5e95f619c4"}
Sep 07 11:30:52.459725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:52.459686    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw" podStartSLOduration=2.292241145 podStartE2EDuration="6.45967248s" totalImagesPullingTime="4.167431335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:30:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:30:52.456675347 +0000 UTC m=+484.404026131" watchObservedRunningTime="2026-09-07 11:30:52.45967248 +0000 UTC m=+484.407023264"
Sep 07 11:30:53.830816 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.830781    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"]
Sep 07 11:30:53.833937 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.833908    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:30:53.837085 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.837061    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bgzkf\""
Sep 07 11:30:53.864194 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.864169    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"]
Sep 07 11:30:53.872009 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.871977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7hbb\" (UniqueName: \"kubernetes.io/projected/0d072482-db94-4757-909d-cb1fdad2e7d5-kube-api-access-l7hbb\") pod \"limitador-operator-controller-manager-6c549b6d85-btzvm\" (UID: \"0d072482-db94-4757-909d-cb1fdad2e7d5\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:30:53.972633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.972606    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7hbb\" (UniqueName: \"kubernetes.io/projected/0d072482-db94-4757-909d-cb1fdad2e7d5-kube-api-access-l7hbb\") pod \"limitador-operator-controller-manager-6c549b6d85-btzvm\" (UID: \"0d072482-db94-4757-909d-cb1fdad2e7d5\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:30:53.984666 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:53.984640    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7hbb\" (UniqueName: \"kubernetes.io/projected/0d072482-db94-4757-909d-cb1fdad2e7d5-kube-api-access-l7hbb\") pod \"limitador-operator-controller-manager-6c549b6d85-btzvm\" (UID: \"0d072482-db94-4757-909d-cb1fdad2e7d5\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:30:54.146152 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:54.146081    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:30:54.421634 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:54.421560    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"]
Sep 07 11:30:54.424818 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:30:54.424793    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d072482_db94_4757_909d_cb1fdad2e7d5.slice/crio-d41c477d9aaf4cfacbfb19c192f3497246bff80f2561361b19a3a378dcbe8614 WatchSource:0}: Error finding container d41c477d9aaf4cfacbfb19c192f3497246bff80f2561361b19a3a378dcbe8614: Status 404 returned error can't find the container with id d41c477d9aaf4cfacbfb19c192f3497246bff80f2561361b19a3a378dcbe8614
Sep 07 11:30:54.448559 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:54.448520    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz" event={"ID":"edf3f957-8a75-445e-8780-be269da59b07","Type":"ContainerStarted","Data":"2ff07215f1a609812a18b199d89184a6573940cacf05ce5e9797c8cf54d98e3d"}
Sep 07 11:30:54.448709 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:54.448683    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:30:54.450061 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:54.450032    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm" event={"ID":"0d072482-db94-4757-909d-cb1fdad2e7d5","Type":"ContainerStarted","Data":"d41c477d9aaf4cfacbfb19c192f3497246bff80f2561361b19a3a378dcbe8614"}
Sep 07 11:30:56.462595 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:56.462503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm" event={"ID":"0d072482-db94-4757-909d-cb1fdad2e7d5","Type":"ContainerStarted","Data":"dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51"}
Sep 07 11:30:56.462595 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:56.462579    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:30:56.479344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:56.479289    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm" podStartSLOduration=1.748745354 podStartE2EDuration="3.479270265s" totalImagesPullingTime="1.730524911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:30:56.47884097 +0000 UTC m=+488.426191754" watchObservedRunningTime="2026-09-07 11:30:56.479270265 +0000 UTC m=+488.426621050"
Sep 07 11:30:56.480369 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:30:56.480331    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz" podStartSLOduration=3.708280813 podStartE2EDuration="5.480317452s" totalImagesPullingTime="1.772036639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:30:54.473146693 +0000 UTC m=+486.420497476" watchObservedRunningTime="2026-09-07 11:30:56.480317452 +0000 UTC m=+488.427668237"
Sep 07 11:31:03.443293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:03.443259    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:05.459498 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.459460    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-4xmxz"
Sep 07 11:31:05.806719 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.806684    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"]
Sep 07 11:31:05.806933 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.806907    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" containerName="manager" containerID="cri-o://3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad" gracePeriod=2
Sep 07 11:31:05.818686 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.818655    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"]
Sep 07 11:31:05.832034 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:05.832000    2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b34e7048-afe4-484f-b90a-65e11ca28206\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-vt4mw\": pods \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:05.840416 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.840388    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"]
Sep 07 11:31:05.840861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.840842    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" containerName="manager"
Sep 07 11:31:05.840861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.840860    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" containerName="manager"
Sep 07 11:31:05.841036 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.840934    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" containerName="manager"
Sep 07 11:31:05.843159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.843139    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:05.844883 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:05.844848    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:05.867410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.867377    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"]
Sep 07 11:31:05.869817 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.869770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a83f2fd-cd65-4dcf-a199-84b575c87a80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5fsdg\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:05.869946 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.869830    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlmgn\" (UniqueName: \"kubernetes.io/projected/7a83f2fd-cd65-4dcf-a199-84b575c87a80-kube-api-access-wlmgn\") pod \"kuadrant-operator-controller-manager-58fc56757c-5fsdg\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:05.872214 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.872193    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"]
Sep 07 11:31:05.872460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.872434    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" containerName="manager" containerID="cri-o://dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51" gracePeriod=2
Sep 07 11:31:05.874482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.874444    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:05.886182 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:05.886149    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:05.888425 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:05.887841    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-btzvm\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:05.888623 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.888529    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"]
Sep 07 11:31:05.895605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.895583    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"]
Sep 07 11:31:05.896016 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.895995    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" containerName="manager"
Sep 07 11:31:05.896016 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.896015    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" containerName="manager"
Sep 07 11:31:05.896184 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.896089    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" containerName="manager"
Sep 07 11:31:05.898141 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.898123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:05.899733 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:05.899708    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:05.911204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.911179    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"]
Sep 07 11:31:05.930164 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:05.930127    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-btzvm\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:05.970617 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.970586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a83f2fd-cd65-4dcf-a199-84b575c87a80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5fsdg\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:05.970760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.970626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlmgn\" (UniqueName: \"kubernetes.io/projected/7a83f2fd-cd65-4dcf-a199-84b575c87a80-kube-api-access-wlmgn\") pod \"kuadrant-operator-controller-manager-58fc56757c-5fsdg\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:05.970760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.970700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d5tz\" (UniqueName: \"kubernetes.io/projected/7854a100-6825-45b0-b860-c4db62d2fe47-kube-api-access-4d5tz\") pod \"limitador-operator-controller-manager-6c549b6d85-d5tkg\" (UID: \"7854a100-6825-45b0-b860-c4db62d2fe47\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:05.970940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.970922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a83f2fd-cd65-4dcf-a199-84b575c87a80-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5fsdg\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:05.985038 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:05.985010    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlmgn\" (UniqueName: \"kubernetes.io/projected/7a83f2fd-cd65-4dcf-a199-84b575c87a80-kube-api-access-wlmgn\") pod \"kuadrant-operator-controller-manager-58fc56757c-5fsdg\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:06.071321 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.071247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d5tz\" (UniqueName: \"kubernetes.io/projected/7854a100-6825-45b0-b860-c4db62d2fe47-kube-api-access-4d5tz\") pod \"limitador-operator-controller-manager-6c549b6d85-d5tkg\" (UID: \"7854a100-6825-45b0-b860-c4db62d2fe47\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:06.102340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.102314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d5tz\" (UniqueName: \"kubernetes.io/projected/7854a100-6825-45b0-b860-c4db62d2fe47-kube-api-access-4d5tz\") pod \"limitador-operator-controller-manager-6c549b6d85-d5tkg\" (UID: \"7854a100-6825-45b0-b860-c4db62d2fe47\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:06.109040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.109019    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:06.111533 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:06.111507    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:06.111811 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.111796    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:06.114126 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:06.114105    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-btzvm\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:06.116156 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:06.116136    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-vt4mw\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:06.118419 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:06.118402    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-btzvm\" is forbidden: User \"system:node:ip-10-0-138-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-45.ec2.internal' and this object" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:06.172060 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.172028    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34e7048-afe4-484f-b90a-65e11ca28206-extensions-socket-volume\") pod \"b34e7048-afe4-484f-b90a-65e11ca28206\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") "
Sep 07 11:31:06.172217 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.172090    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7hbb\" (UniqueName: \"kubernetes.io/projected/0d072482-db94-4757-909d-cb1fdad2e7d5-kube-api-access-l7hbb\") pod \"0d072482-db94-4757-909d-cb1fdad2e7d5\" (UID: \"0d072482-db94-4757-909d-cb1fdad2e7d5\") "
Sep 07 11:31:06.172217 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.172150    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8njqc\" (UniqueName: \"kubernetes.io/projected/b34e7048-afe4-484f-b90a-65e11ca28206-kube-api-access-8njqc\") pod \"b34e7048-afe4-484f-b90a-65e11ca28206\" (UID: \"b34e7048-afe4-484f-b90a-65e11ca28206\") "
Sep 07 11:31:06.172596 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.172566    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34e7048-afe4-484f-b90a-65e11ca28206-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b34e7048-afe4-484f-b90a-65e11ca28206" (UID: "b34e7048-afe4-484f-b90a-65e11ca28206"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:31:06.174283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.174261    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d072482-db94-4757-909d-cb1fdad2e7d5-kube-api-access-l7hbb" (OuterVolumeSpecName: "kube-api-access-l7hbb") pod "0d072482-db94-4757-909d-cb1fdad2e7d5" (UID: "0d072482-db94-4757-909d-cb1fdad2e7d5"). InnerVolumeSpecName "kube-api-access-l7hbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:31:06.174283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.174274    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34e7048-afe4-484f-b90a-65e11ca28206-kube-api-access-8njqc" (OuterVolumeSpecName: "kube-api-access-8njqc") pod "b34e7048-afe4-484f-b90a-65e11ca28206" (UID: "b34e7048-afe4-484f-b90a-65e11ca28206"). InnerVolumeSpecName "kube-api-access-8njqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:31:06.253753 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.253703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:06.259359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.259337    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:06.273522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.273482    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7hbb\" (UniqueName: \"kubernetes.io/projected/0d072482-db94-4757-909d-cb1fdad2e7d5-kube-api-access-l7hbb\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:31:06.273522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.273517    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8njqc\" (UniqueName: \"kubernetes.io/projected/b34e7048-afe4-484f-b90a-65e11ca28206-kube-api-access-8njqc\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:31:06.273679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.273534    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34e7048-afe4-484f-b90a-65e11ca28206-extensions-socket-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:31:06.497282 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.497246    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d072482-db94-4757-909d-cb1fdad2e7d5" path="/var/lib/kubelet/pods/0d072482-db94-4757-909d-cb1fdad2e7d5/volumes"
Sep 07 11:31:06.497628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.497573    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34e7048-afe4-484f-b90a-65e11ca28206" path="/var/lib/kubelet/pods/b34e7048-afe4-484f-b90a-65e11ca28206/volumes"
Sep 07 11:31:06.509667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.509620    2560 generic.go:356] "Generic (PLEG): container finished" podID="0d072482-db94-4757-909d-cb1fdad2e7d5" containerID="dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51" exitCode=0
Sep 07 11:31:06.509805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.509676    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-btzvm"
Sep 07 11:31:06.509805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.509714    2560 scope.go:117] "RemoveContainer" containerID="dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51"
Sep 07 11:31:06.516850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.516826    2560 generic.go:356] "Generic (PLEG): container finished" podID="b34e7048-afe4-484f-b90a-65e11ca28206" containerID="3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad" exitCode=0
Sep 07 11:31:06.516977 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.516958    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vt4mw"
Sep 07 11:31:06.523508 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.523490    2560 scope.go:117] "RemoveContainer" containerID="dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51"
Sep 07 11:31:06.523770 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:06.523753    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51\": container with ID starting with dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51 not found: ID does not exist" containerID="dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51"
Sep 07 11:31:06.523815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.523777    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51"} err="failed to get container status \"dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51\": rpc error: code = NotFound desc = could not find container \"dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51\": container with ID starting with dfe82c1fef5a80c2e157850694a76d0e5c7c609e13569724e1875d9780bdeb51 not found: ID does not exist"
Sep 07 11:31:06.523815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.523795    2560 scope.go:117] "RemoveContainer" containerID="3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad"
Sep 07 11:31:06.531174 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.531158    2560 scope.go:117] "RemoveContainer" containerID="3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad"
Sep 07 11:31:06.531436 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:31:06.531420    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad\": container with ID starting with 3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad not found: ID does not exist" containerID="3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad"
Sep 07 11:31:06.531492 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.531441    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad"} err="failed to get container status \"3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad\": rpc error: code = NotFound desc = could not find container \"3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad\": container with ID starting with 3bcbb3b6b84c369b8f1ba3d01dd25aca606ba5dcbac5ee7cd6cf91016a7d1bad not found: ID does not exist"
Sep 07 11:31:06.540626 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.540603    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"]
Sep 07 11:31:06.542247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:06.542228    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"]
Sep 07 11:31:06.544847 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:31:06.544823    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a83f2fd_cd65_4dcf_a199_84b575c87a80.slice/crio-966ea0559854a90c64875f4e2565244339d81ef7afd2f3bdc23522c7dfd96a62 WatchSource:0}: Error finding container 966ea0559854a90c64875f4e2565244339d81ef7afd2f3bdc23522c7dfd96a62: Status 404 returned error can't find the container with id 966ea0559854a90c64875f4e2565244339d81ef7afd2f3bdc23522c7dfd96a62
Sep 07 11:31:06.545541 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:31:06.545524    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7854a100_6825_45b0_b860_c4db62d2fe47.slice/crio-9b1e98c737fa1e7ce1c4104a06c135e0ba9f36f45910be15ee1cc2067e1f785f WatchSource:0}: Error finding container 9b1e98c737fa1e7ce1c4104a06c135e0ba9f36f45910be15ee1cc2067e1f785f: Status 404 returned error can't find the container with id 9b1e98c737fa1e7ce1c4104a06c135e0ba9f36f45910be15ee1cc2067e1f785f
Sep 07 11:31:07.525107 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.525071    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg" event={"ID":"7a83f2fd-cd65-4dcf-a199-84b575c87a80","Type":"ContainerStarted","Data":"f26a6e2253b318e6da2b5f7993dc229eadcfb267a09effa0e83272a270b6cf9e"}
Sep 07 11:31:07.525542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.525126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg" event={"ID":"7a83f2fd-cd65-4dcf-a199-84b575c87a80","Type":"ContainerStarted","Data":"966ea0559854a90c64875f4e2565244339d81ef7afd2f3bdc23522c7dfd96a62"}
Sep 07 11:31:07.525542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.525154    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:07.526965 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.526931    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg" event={"ID":"7854a100-6825-45b0-b860-c4db62d2fe47","Type":"ContainerStarted","Data":"a694100706f43cdd4b1bbecbb4054f3ab68d0b78dd43d67aac7e396ab19cef49"}
Sep 07 11:31:07.527088 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.526985    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg" event={"ID":"7854a100-6825-45b0-b860-c4db62d2fe47","Type":"ContainerStarted","Data":"9b1e98c737fa1e7ce1c4104a06c135e0ba9f36f45910be15ee1cc2067e1f785f"}
Sep 07 11:31:07.527088 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.527017    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:07.547370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.547322    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg" podStartSLOduration=2.547304705 podStartE2EDuration="2.547304705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:31:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:31:07.546654003 +0000 UTC m=+499.494004786" watchObservedRunningTime="2026-09-07 11:31:07.547304705 +0000 UTC m=+499.494655488"
Sep 07 11:31:07.566681 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:07.566638    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg" podStartSLOduration=2.566622864 podStartE2EDuration="2.566622864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:31:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:31:07.565193863 +0000 UTC m=+499.512544647" watchObservedRunningTime="2026-09-07 11:31:07.566622864 +0000 UTC m=+499.513973647"
Sep 07 11:31:18.535043 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:18.535011    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-d5tkg"
Sep 07 11:31:18.535477 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:18.535093    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:21.397784 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.397731    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"]
Sep 07 11:31:21.398221 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.397968    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg" podUID="7a83f2fd-cd65-4dcf-a199-84b575c87a80" containerName="manager" containerID="cri-o://f26a6e2253b318e6da2b5f7993dc229eadcfb267a09effa0e83272a270b6cf9e" gracePeriod=10
Sep 07 11:31:21.593654 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.593623    2560 generic.go:356] "Generic (PLEG): container finished" podID="7a83f2fd-cd65-4dcf-a199-84b575c87a80" containerID="f26a6e2253b318e6da2b5f7993dc229eadcfb267a09effa0e83272a270b6cf9e" exitCode=0
Sep 07 11:31:21.593813 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.593696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg" event={"ID":"7a83f2fd-cd65-4dcf-a199-84b575c87a80","Type":"ContainerDied","Data":"f26a6e2253b318e6da2b5f7993dc229eadcfb267a09effa0e83272a270b6cf9e"}
Sep 07 11:31:21.646886 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.646864    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:21.813450 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.813415    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlmgn\" (UniqueName: \"kubernetes.io/projected/7a83f2fd-cd65-4dcf-a199-84b575c87a80-kube-api-access-wlmgn\") pod \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") "
Sep 07 11:31:21.813634 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.813482    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a83f2fd-cd65-4dcf-a199-84b575c87a80-extensions-socket-volume\") pod \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\" (UID: \"7a83f2fd-cd65-4dcf-a199-84b575c87a80\") "
Sep 07 11:31:21.813880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.813850    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a83f2fd-cd65-4dcf-a199-84b575c87a80-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7a83f2fd-cd65-4dcf-a199-84b575c87a80" (UID: "7a83f2fd-cd65-4dcf-a199-84b575c87a80"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:31:21.815584 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.815545    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a83f2fd-cd65-4dcf-a199-84b575c87a80-kube-api-access-wlmgn" (OuterVolumeSpecName: "kube-api-access-wlmgn") pod "7a83f2fd-cd65-4dcf-a199-84b575c87a80" (UID: "7a83f2fd-cd65-4dcf-a199-84b575c87a80"). InnerVolumeSpecName "kube-api-access-wlmgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:31:21.914070 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.914035    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlmgn\" (UniqueName: \"kubernetes.io/projected/7a83f2fd-cd65-4dcf-a199-84b575c87a80-kube-api-access-wlmgn\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:31:21.914070 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:21.914065    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a83f2fd-cd65-4dcf-a199-84b575c87a80-extensions-socket-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:31:22.599755 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:22.599703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg" event={"ID":"7a83f2fd-cd65-4dcf-a199-84b575c87a80","Type":"ContainerDied","Data":"966ea0559854a90c64875f4e2565244339d81ef7afd2f3bdc23522c7dfd96a62"}
Sep 07 11:31:22.600140 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:22.599789    2560 scope.go:117] "RemoveContainer" containerID="f26a6e2253b318e6da2b5f7993dc229eadcfb267a09effa0e83272a270b6cf9e"
Sep 07 11:31:22.600140 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:22.599718    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"
Sep 07 11:31:22.619858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:22.619832    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"]
Sep 07 11:31:22.623304 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:22.623285    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5fsdg"]
Sep 07 11:31:24.497028 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:24.496994    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a83f2fd-cd65-4dcf-a199-84b575c87a80" path="/var/lib/kubelet/pods/7a83f2fd-cd65-4dcf-a199-84b575c87a80/volumes"
Sep 07 11:31:37.797849 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.797818    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"]
Sep 07 11:31:37.798241 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.798177    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a83f2fd-cd65-4dcf-a199-84b575c87a80" containerName="manager"
Sep 07 11:31:37.798241 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.798190    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a83f2fd-cd65-4dcf-a199-84b575c87a80" containerName="manager"
Sep 07 11:31:37.798312 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.798258    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a83f2fd-cd65-4dcf-a199-84b575c87a80" containerName="manager"
Sep 07 11:31:37.801540 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.801522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.803783 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.803758    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-skrj7\""
Sep 07 11:31:37.813080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.813054    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"]
Sep 07 11:31:37.853469 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853434    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a89314-0821-4934-a99c-74b3a9f3e3eb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853788 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853637    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853788 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853654    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjpl4\" (UniqueName: \"kubernetes.io/projected/92a89314-0821-4934-a99c-74b3a9f3e3eb-kube-api-access-pjpl4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853788 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853675    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.853788 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.853694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954063    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a89314-0821-4934-a99c-74b3a9f3e3eb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954152    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjpl4\" (UniqueName: \"kubernetes.io/projected/92a89314-0821-4934-a99c-74b3a9f3e3eb-kube-api-access-pjpl4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954221    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954289    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954700    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.954983 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.954959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a89314-0821-4934-a99c-74b3a9f3e3eb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.956523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.956506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.956805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.956787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.977394 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.977365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjpl4\" (UniqueName: \"kubernetes.io/projected/92a89314-0821-4934-a99c-74b3a9f3e3eb-kube-api-access-pjpl4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:37.977394 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:37.977373    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a89314-0821-4934-a99c-74b3a9f3e3eb-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qnwdb\" (UID: \"92a89314-0821-4934-a99c-74b3a9f3e3eb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:38.112378 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.112288    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:38.386634 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.386571    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"]
Sep 07 11:31:38.388925 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:31:38.388893    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a89314_0821_4934_a99c_74b3a9f3e3eb.slice/crio-f6d710e7efcabe57ed8ab2726dbf915ed2480bf47bcf1a0aa7b86c39c8a1836f WatchSource:0}: Error finding container f6d710e7efcabe57ed8ab2726dbf915ed2480bf47bcf1a0aa7b86c39c8a1836f: Status 404 returned error can't find the container with id f6d710e7efcabe57ed8ab2726dbf915ed2480bf47bcf1a0aa7b86c39c8a1836f
Sep 07 11:31:38.390861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.390826    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:31:38.390967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.390902    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:31:38.390967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.390944    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:31:38.676794 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.676686    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" event={"ID":"92a89314-0821-4934-a99c-74b3a9f3e3eb","Type":"ContainerStarted","Data":"fc93500336e4b3adbe6105309d287746f49145a6e9b6c6598f90696e6869ff18"}
Sep 07 11:31:38.676794 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.676758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" event={"ID":"92a89314-0821-4934-a99c-74b3a9f3e3eb","Type":"ContainerStarted","Data":"f6d710e7efcabe57ed8ab2726dbf915ed2480bf47bcf1a0aa7b86c39c8a1836f"}
Sep 07 11:31:38.699304 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:38.699259    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" podStartSLOduration=1.699242564 podStartE2EDuration="1.699242564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:31:38.698134832 +0000 UTC m=+530.645485628" watchObservedRunningTime="2026-09-07 11:31:38.699242564 +0000 UTC m=+530.646593399"
Sep 07 11:31:39.113223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:39.113185    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:39.117843 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:39.117816    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:39.685125 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:39.683270    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:39.687710 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:39.687684    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:31:54.198911 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.198877    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:31:54.202358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.202340    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.204636 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.204612    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xk74w\""
Sep 07 11:31:54.204771 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.204661    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 11:31:54.211362 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.211337    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:31:54.298264 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.298227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:31:54.299725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.299703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e82efa73-c4dd-4352-bf21-003a2c35f17e-config-file\") pod \"limitador-limitador-859c4d96d6-742tg\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.299848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.299756    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzzb\" (UniqueName: \"kubernetes.io/projected/e82efa73-c4dd-4352-bf21-003a2c35f17e-kube-api-access-dqzzb\") pod \"limitador-limitador-859c4d96d6-742tg\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.400963 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.400925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e82efa73-c4dd-4352-bf21-003a2c35f17e-config-file\") pod \"limitador-limitador-859c4d96d6-742tg\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.401160 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.400985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzzb\" (UniqueName: \"kubernetes.io/projected/e82efa73-c4dd-4352-bf21-003a2c35f17e-kube-api-access-dqzzb\") pod \"limitador-limitador-859c4d96d6-742tg\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.401862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.401825    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e82efa73-c4dd-4352-bf21-003a2c35f17e-config-file\") pod \"limitador-limitador-859c4d96d6-742tg\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.411399 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.411371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzzb\" (UniqueName: \"kubernetes.io/projected/e82efa73-c4dd-4352-bf21-003a2c35f17e-kube-api-access-dqzzb\") pod \"limitador-limitador-859c4d96d6-742tg\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.513447 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.513416    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:54.791245 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:54.791212    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:31:54.792945 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:31:54.792917    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82efa73_c4dd_4352_bf21_003a2c35f17e.slice/crio-1ceddf9a8273d927bb52ad2f89371365da1eb09b60f5d85af0b234d453c4a8d6 WatchSource:0}: Error finding container 1ceddf9a8273d927bb52ad2f89371365da1eb09b60f5d85af0b234d453c4a8d6: Status 404 returned error can't find the container with id 1ceddf9a8273d927bb52ad2f89371365da1eb09b60f5d85af0b234d453c4a8d6
Sep 07 11:31:55.268928 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.268892    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gnk8c"]
Sep 07 11:31:55.272618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.272589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:31:55.275429 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.275410    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wr2st\""
Sep 07 11:31:55.280277 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.280017    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gnk8c"]
Sep 07 11:31:55.308042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.308010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6h8g\" (UniqueName: \"kubernetes.io/projected/c9207675-217c-460a-b147-7d145e9ff1fe-kube-api-access-k6h8g\") pod \"authorino-c548b5d7d-gnk8c\" (UID: \"c9207675-217c-460a-b147-7d145e9ff1fe\") " pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:31:55.408630 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.408597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6h8g\" (UniqueName: \"kubernetes.io/projected/c9207675-217c-460a-b147-7d145e9ff1fe-kube-api-access-k6h8g\") pod \"authorino-c548b5d7d-gnk8c\" (UID: \"c9207675-217c-460a-b147-7d145e9ff1fe\") " pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:31:55.421378 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.421351    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6h8g\" (UniqueName: \"kubernetes.io/projected/c9207675-217c-460a-b147-7d145e9ff1fe-kube-api-access-k6h8g\") pod \"authorino-c548b5d7d-gnk8c\" (UID: \"c9207675-217c-460a-b147-7d145e9ff1fe\") " pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:31:55.583128 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.583052    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:31:55.764140 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.764107    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg" event={"ID":"e82efa73-c4dd-4352-bf21-003a2c35f17e","Type":"ContainerStarted","Data":"1ceddf9a8273d927bb52ad2f89371365da1eb09b60f5d85af0b234d453c4a8d6"}
Sep 07 11:31:55.864258 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:55.864191    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gnk8c"]
Sep 07 11:31:55.866811 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:31:55.866780    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9207675_217c_460a_b147_7d145e9ff1fe.slice/crio-c25c7d2a6b1b5ef0e7c6043e70996be45e8fe8221ecdd9492aa74815de83c702 WatchSource:0}: Error finding container c25c7d2a6b1b5ef0e7c6043e70996be45e8fe8221ecdd9492aa74815de83c702: Status 404 returned error can't find the container with id c25c7d2a6b1b5ef0e7c6043e70996be45e8fe8221ecdd9492aa74815de83c702
Sep 07 11:31:56.772206 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:56.772162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg" event={"ID":"e82efa73-c4dd-4352-bf21-003a2c35f17e","Type":"ContainerStarted","Data":"79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196"}
Sep 07 11:31:56.772688 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:56.772400    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:31:56.774080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:56.774048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gnk8c" event={"ID":"c9207675-217c-460a-b147-7d145e9ff1fe","Type":"ContainerStarted","Data":"c25c7d2a6b1b5ef0e7c6043e70996be45e8fe8221ecdd9492aa74815de83c702"}
Sep 07 11:31:56.791607 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:56.791535    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg" podStartSLOduration=1.412645896 podStartE2EDuration="2.791516653s" totalImagesPullingTime="1.378870757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:31:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:31:56.791041138 +0000 UTC m=+548.738391924" watchObservedRunningTime="2026-09-07 11:31:56.791516653 +0000 UTC m=+548.738867440"
Sep 07 11:31:57.782678 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:57.782645    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gnk8c" event={"ID":"c9207675-217c-460a-b147-7d145e9ff1fe","Type":"ContainerStarted","Data":"6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46"}
Sep 07 11:31:57.801572 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:31:57.801489    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-gnk8c" podStartSLOduration=1.124818348 podStartE2EDuration="2.801472421s" totalImagesPullingTime="1.676654073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:31:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:31:57.800393838 +0000 UTC m=+549.747744625" watchObservedRunningTime="2026-09-07 11:31:57.801472421 +0000 UTC m=+549.748823204"
Sep 07 11:32:07.647851 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:07.647812    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:32:07.648268 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:07.648032    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg" podUID="e82efa73-c4dd-4352-bf21-003a2c35f17e" containerName="limitador" containerID="cri-o://79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196" gracePeriod=30
Sep 07 11:32:07.648784 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:07.648728    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:32:08.189800 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.189775    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:32:08.328756 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.328711    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e82efa73-c4dd-4352-bf21-003a2c35f17e-config-file\") pod \"e82efa73-c4dd-4352-bf21-003a2c35f17e\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") "
Sep 07 11:32:08.328928 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.328895    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqzzb\" (UniqueName: \"kubernetes.io/projected/e82efa73-c4dd-4352-bf21-003a2c35f17e-kube-api-access-dqzzb\") pod \"e82efa73-c4dd-4352-bf21-003a2c35f17e\" (UID: \"e82efa73-c4dd-4352-bf21-003a2c35f17e\") "
Sep 07 11:32:08.329061 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.329038    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e82efa73-c4dd-4352-bf21-003a2c35f17e-config-file" (OuterVolumeSpecName: "config-file") pod "e82efa73-c4dd-4352-bf21-003a2c35f17e" (UID: "e82efa73-c4dd-4352-bf21-003a2c35f17e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:32:08.330911 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.330879    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e82efa73-c4dd-4352-bf21-003a2c35f17e-kube-api-access-dqzzb" (OuterVolumeSpecName: "kube-api-access-dqzzb") pod "e82efa73-c4dd-4352-bf21-003a2c35f17e" (UID: "e82efa73-c4dd-4352-bf21-003a2c35f17e"). InnerVolumeSpecName "kube-api-access-dqzzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:32:08.430405 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.430371    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e82efa73-c4dd-4352-bf21-003a2c35f17e-config-file\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:08.430405 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.430397    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqzzb\" (UniqueName: \"kubernetes.io/projected/e82efa73-c4dd-4352-bf21-003a2c35f17e-kube-api-access-dqzzb\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:08.832289 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.832258    2560 generic.go:356] "Generic (PLEG): container finished" podID="e82efa73-c4dd-4352-bf21-003a2c35f17e" containerID="79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196" exitCode=0
Sep 07 11:32:08.832693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.832307    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg"
Sep 07 11:32:08.832693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.832350    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg" event={"ID":"e82efa73-c4dd-4352-bf21-003a2c35f17e","Type":"ContainerDied","Data":"79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196"}
Sep 07 11:32:08.832693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.832401    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-742tg" event={"ID":"e82efa73-c4dd-4352-bf21-003a2c35f17e","Type":"ContainerDied","Data":"1ceddf9a8273d927bb52ad2f89371365da1eb09b60f5d85af0b234d453c4a8d6"}
Sep 07 11:32:08.832693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.832425    2560 scope.go:117] "RemoveContainer" containerID="79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196"
Sep 07 11:32:08.840688 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.840674    2560 scope.go:117] "RemoveContainer" containerID="79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196"
Sep 07 11:32:08.840955 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:08.840936    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196\": container with ID starting with 79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196 not found: ID does not exist" containerID="79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196"
Sep 07 11:32:08.841025 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.840962    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196"} err="failed to get container status \"79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196\": rpc error: code = NotFound desc = could not find container \"79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196\": container with ID starting with 79362c41c87ae8e1b758fed133590f369c2a3814f52249a51e64aa6146aa6196 not found: ID does not exist"
Sep 07 11:32:08.848604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.848582    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:32:08.852376 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:08.852356    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-742tg"]
Sep 07 11:32:10.497447 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:10.497410    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e82efa73-c4dd-4352-bf21-003a2c35f17e" path="/var/lib/kubelet/pods/e82efa73-c4dd-4352-bf21-003a2c35f17e/volumes"
Sep 07 11:32:11.969564 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.969526    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"]
Sep 07 11:32:11.969953 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.969894    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e82efa73-c4dd-4352-bf21-003a2c35f17e" containerName="limitador"
Sep 07 11:32:11.969953 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.969906    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e82efa73-c4dd-4352-bf21-003a2c35f17e" containerName="limitador"
Sep 07 11:32:11.970035 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.969958    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e82efa73-c4dd-4352-bf21-003a2c35f17e" containerName="limitador"
Sep 07 11:32:11.972701 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.972683    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:11.975085 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.975063    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 11:32:11.975335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.975065    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 07 11:32:11.975690 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.975672    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 11:32:11.975791 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.975671    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zxdn7\""
Sep 07 11:32:11.983607 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:11.983586    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"]
Sep 07 11:32:12.063890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.063857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5b48c858-4ea8-4268-9c58-568ba90f312c-data\") pod \"postgres-69fb4594cf-sjzrr\" (UID: \"5b48c858-4ea8-4268-9c58-568ba90f312c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.064045 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.063900    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27z9n\" (UniqueName: \"kubernetes.io/projected/5b48c858-4ea8-4268-9c58-568ba90f312c-kube-api-access-27z9n\") pod \"postgres-69fb4594cf-sjzrr\" (UID: \"5b48c858-4ea8-4268-9c58-568ba90f312c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.165040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.165004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5b48c858-4ea8-4268-9c58-568ba90f312c-data\") pod \"postgres-69fb4594cf-sjzrr\" (UID: \"5b48c858-4ea8-4268-9c58-568ba90f312c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.165230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.165055    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27z9n\" (UniqueName: \"kubernetes.io/projected/5b48c858-4ea8-4268-9c58-568ba90f312c-kube-api-access-27z9n\") pod \"postgres-69fb4594cf-sjzrr\" (UID: \"5b48c858-4ea8-4268-9c58-568ba90f312c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.165372 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.165346    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5b48c858-4ea8-4268-9c58-568ba90f312c-data\") pod \"postgres-69fb4594cf-sjzrr\" (UID: \"5b48c858-4ea8-4268-9c58-568ba90f312c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.173196 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.173171    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27z9n\" (UniqueName: \"kubernetes.io/projected/5b48c858-4ea8-4268-9c58-568ba90f312c-kube-api-access-27z9n\") pod \"postgres-69fb4594cf-sjzrr\" (UID: \"5b48c858-4ea8-4268-9c58-568ba90f312c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.285830 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.285796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:12.555432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.555394    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"]
Sep 07 11:32:12.557871 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:32:12.557841    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b48c858_4ea8_4268_9c58_568ba90f312c.slice/crio-f1292c2ef185e370fd650c2c86c6b973e7f4cb43a12d9974cc418d9d9b8a16a6 WatchSource:0}: Error finding container f1292c2ef185e370fd650c2c86c6b973e7f4cb43a12d9974cc418d9d9b8a16a6: Status 404 returned error can't find the container with id f1292c2ef185e370fd650c2c86c6b973e7f4cb43a12d9974cc418d9d9b8a16a6
Sep 07 11:32:12.853458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:12.853366    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr" event={"ID":"5b48c858-4ea8-4268-9c58-568ba90f312c","Type":"ContainerStarted","Data":"f1292c2ef185e370fd650c2c86c6b973e7f4cb43a12d9974cc418d9d9b8a16a6"}
Sep 07 11:32:17.880908 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:17.880864    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr" event={"ID":"5b48c858-4ea8-4268-9c58-568ba90f312c","Type":"ContainerStarted","Data":"6255a904fecc5ff99347a7631cc24f7143e7e1f55d663da160281949c314fff5"}
Sep 07 11:32:17.881328 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:17.881031    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:17.898158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:17.897968    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr" podStartSLOduration=2.218202902 podStartE2EDuration="6.897952268s" totalImagesPullingTime="4.679749366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:32:17.897821557 +0000 UTC m=+569.845172342" watchObservedRunningTime="2026-09-07 11:32:17.897952268 +0000 UTC m=+569.845303057"
Sep 07 11:32:23.914381 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:23.914350    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sjzrr"
Sep 07 11:32:24.537621 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.537586    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-wqp98"]
Sep 07 11:32:24.541398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.541371    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:24.553498 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.553470    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-wqp98"]
Sep 07 11:32:24.575021 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.574990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv6mb\" (UniqueName: \"kubernetes.io/projected/48c7fe30-087b-4421-897e-32a9e2e1c7bc-kube-api-access-gv6mb\") pod \"authorino-5dc7ccb5f-wqp98\" (UID: \"48c7fe30-087b-4421-897e-32a9e2e1c7bc\") " pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:24.675972 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.675935    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv6mb\" (UniqueName: \"kubernetes.io/projected/48c7fe30-087b-4421-897e-32a9e2e1c7bc-kube-api-access-gv6mb\") pod \"authorino-5dc7ccb5f-wqp98\" (UID: \"48c7fe30-087b-4421-897e-32a9e2e1c7bc\") " pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:24.684868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.684842    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv6mb\" (UniqueName: \"kubernetes.io/projected/48c7fe30-087b-4421-897e-32a9e2e1c7bc-kube-api-access-gv6mb\") pod \"authorino-5dc7ccb5f-wqp98\" (UID: \"48c7fe30-087b-4421-897e-32a9e2e1c7bc\") " pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:24.794922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.794829    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-wqp98"]
Sep 07 11:32:24.795120 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.795107    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:24.823242 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.823211    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c647dd955-tqnbj"]
Sep 07 11:32:24.826416 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.826395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:24.836584 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.836557    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c647dd955-tqnbj"]
Sep 07 11:32:24.864327 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.864291    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c647dd955-tqnbj"]
Sep 07 11:32:24.864619 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:24.864584    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-n6hzb], unattached volumes=[], failed to process volumes=[kube-api-access-n6hzb]: context canceled" pod="kuadrant-system/authorino-6c647dd955-tqnbj" podUID="0f784bb7-a6ac-43be-8e6c-0fed7d753277"
Sep 07 11:32:24.877079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.877044    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6hzb\" (UniqueName: \"kubernetes.io/projected/0f784bb7-a6ac-43be-8e6c-0fed7d753277-kube-api-access-n6hzb\") pod \"authorino-6c647dd955-tqnbj\" (UID: \"0f784bb7-a6ac-43be-8e6c-0fed7d753277\") " pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:24.897695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.897660    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9d9b666fc-2bvgl"]
Sep 07 11:32:24.900108 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.900093    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:24.902396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.902378    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 07 11:32:24.908902 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.908878    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9d9b666fc-2bvgl"]
Sep 07 11:32:24.911813 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.911791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:24.917230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.917212    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:24.977357 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.977322    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6hzb\" (UniqueName: \"kubernetes.io/projected/0f784bb7-a6ac-43be-8e6c-0fed7d753277-kube-api-access-n6hzb\") pod \"authorino-6c647dd955-tqnbj\" (UID: \"0f784bb7-a6ac-43be-8e6c-0fed7d753277\") " pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:24.977540 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.977371    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-kube-api-access-59sjh\") pod \"authorino-9d9b666fc-2bvgl\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") " pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:24.977540 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.977394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-tls-cert\") pod \"authorino-9d9b666fc-2bvgl\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") " pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:24.985294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:24.985262    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6hzb\" (UniqueName: \"kubernetes.io/projected/0f784bb7-a6ac-43be-8e6c-0fed7d753277-kube-api-access-n6hzb\") pod \"authorino-6c647dd955-tqnbj\" (UID: \"0f784bb7-a6ac-43be-8e6c-0fed7d753277\") " pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:25.066726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.066646    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-wqp98"]
Sep 07 11:32:25.069484 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:32:25.069458    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48c7fe30_087b_4421_897e_32a9e2e1c7bc.slice/crio-38f4616dd9379a3d6c96a97e1f6064e4698e0223e4ae4fe5a7012e5ed3e74362 WatchSource:0}: Error finding container 38f4616dd9379a3d6c96a97e1f6064e4698e0223e4ae4fe5a7012e5ed3e74362: Status 404 returned error can't find the container with id 38f4616dd9379a3d6c96a97e1f6064e4698e0223e4ae4fe5a7012e5ed3e74362
Sep 07 11:32:25.077951 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.077928    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6hzb\" (UniqueName: \"kubernetes.io/projected/0f784bb7-a6ac-43be-8e6c-0fed7d753277-kube-api-access-n6hzb\") pod \"0f784bb7-a6ac-43be-8e6c-0fed7d753277\" (UID: \"0f784bb7-a6ac-43be-8e6c-0fed7d753277\") "
Sep 07 11:32:25.078106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.078088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-kube-api-access-59sjh\") pod \"authorino-9d9b666fc-2bvgl\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") " pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:25.078180 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.078134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-tls-cert\") pod \"authorino-9d9b666fc-2bvgl\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") " pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:25.079908 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.079887    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f784bb7-a6ac-43be-8e6c-0fed7d753277-kube-api-access-n6hzb" (OuterVolumeSpecName: "kube-api-access-n6hzb") pod "0f784bb7-a6ac-43be-8e6c-0fed7d753277" (UID: "0f784bb7-a6ac-43be-8e6c-0fed7d753277"). InnerVolumeSpecName "kube-api-access-n6hzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:32:25.080336 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.080317    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-tls-cert\") pod \"authorino-9d9b666fc-2bvgl\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") " pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:25.085981 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.085959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-kube-api-access-59sjh\") pod \"authorino-9d9b666fc-2bvgl\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") " pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:25.179026 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.178992    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6hzb\" (UniqueName: \"kubernetes.io/projected/0f784bb7-a6ac-43be-8e6c-0fed7d753277-kube-api-access-n6hzb\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:25.209907 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.209875    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:32:25.479156 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.479096    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9d9b666fc-2bvgl"]
Sep 07 11:32:25.481216 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:32:25.481190    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddadd9b5a_e5be_4a79_a84a_fdd5ed6fc446.slice/crio-266bd865dc2952539c7aa51bb0ebab6f1e4c13cb7124a057cbb627b20d294a43 WatchSource:0}: Error finding container 266bd865dc2952539c7aa51bb0ebab6f1e4c13cb7124a057cbb627b20d294a43: Status 404 returned error can't find the container with id 266bd865dc2952539c7aa51bb0ebab6f1e4c13cb7124a057cbb627b20d294a43
Sep 07 11:32:25.919224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.919101    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98" event={"ID":"48c7fe30-087b-4421-897e-32a9e2e1c7bc","Type":"ContainerStarted","Data":"d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7"}
Sep 07 11:32:25.919224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.919121    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98" podUID="48c7fe30-087b-4421-897e-32a9e2e1c7bc" containerName="authorino" containerID="cri-o://d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7" gracePeriod=30
Sep 07 11:32:25.919224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.919170    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98" event={"ID":"48c7fe30-087b-4421-897e-32a9e2e1c7bc","Type":"ContainerStarted","Data":"38f4616dd9379a3d6c96a97e1f6064e4698e0223e4ae4fe5a7012e5ed3e74362"}
Sep 07 11:32:25.920646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.920625    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c647dd955-tqnbj"
Sep 07 11:32:25.920769 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.920650    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9b666fc-2bvgl" event={"ID":"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446","Type":"ContainerStarted","Data":"266bd865dc2952539c7aa51bb0ebab6f1e4c13cb7124a057cbb627b20d294a43"}
Sep 07 11:32:25.935353 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:25.935305    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98" podStartSLOduration=1.429662216 podStartE2EDuration="1.935286724s" totalImagesPullingTime="505.624508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:32:25.93417946 +0000 UTC m=+577.881530245" watchObservedRunningTime="2026-09-07 11:32:25.935286724 +0000 UTC m=+577.882637509"
Sep 07 11:32:26.006778 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.006724    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c647dd955-tqnbj"]
Sep 07 11:32:26.010607 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.010584    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c647dd955-tqnbj"]
Sep 07 11:32:26.179919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.179892    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:26.186029 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.186006    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv6mb\" (UniqueName: \"kubernetes.io/projected/48c7fe30-087b-4421-897e-32a9e2e1c7bc-kube-api-access-gv6mb\") pod \"48c7fe30-087b-4421-897e-32a9e2e1c7bc\" (UID: \"48c7fe30-087b-4421-897e-32a9e2e1c7bc\") "
Sep 07 11:32:26.187900 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.187878    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c7fe30-087b-4421-897e-32a9e2e1c7bc-kube-api-access-gv6mb" (OuterVolumeSpecName: "kube-api-access-gv6mb") pod "48c7fe30-087b-4421-897e-32a9e2e1c7bc" (UID: "48c7fe30-087b-4421-897e-32a9e2e1c7bc"). InnerVolumeSpecName "kube-api-access-gv6mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:32:26.287137 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.287109    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv6mb\" (UniqueName: \"kubernetes.io/projected/48c7fe30-087b-4421-897e-32a9e2e1c7bc-kube-api-access-gv6mb\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:26.497018 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.496988    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f784bb7-a6ac-43be-8e6c-0fed7d753277" path="/var/lib/kubelet/pods/0f784bb7-a6ac-43be-8e6c-0fed7d753277/volumes"
Sep 07 11:32:26.926847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.926758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9b666fc-2bvgl" event={"ID":"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446","Type":"ContainerStarted","Data":"7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c"}
Sep 07 11:32:26.927997 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.927970    2560 generic.go:356] "Generic (PLEG): container finished" podID="48c7fe30-087b-4421-897e-32a9e2e1c7bc" containerID="d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7" exitCode=0
Sep 07 11:32:26.928102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.928003    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98"
Sep 07 11:32:26.928102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.928012    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98" event={"ID":"48c7fe30-087b-4421-897e-32a9e2e1c7bc","Type":"ContainerDied","Data":"d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7"}
Sep 07 11:32:26.928102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.928034    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-wqp98" event={"ID":"48c7fe30-087b-4421-897e-32a9e2e1c7bc","Type":"ContainerDied","Data":"38f4616dd9379a3d6c96a97e1f6064e4698e0223e4ae4fe5a7012e5ed3e74362"}
Sep 07 11:32:26.928102 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.928050    2560 scope.go:117] "RemoveContainer" containerID="d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7"
Sep 07 11:32:26.935986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.935967    2560 scope.go:117] "RemoveContainer" containerID="d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7"
Sep 07 11:32:26.936269 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:26.936249    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7\": container with ID starting with d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7 not found: ID does not exist" containerID="d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7"
Sep 07 11:32:26.936352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.936273    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7"} err="failed to get container status \"d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7\": rpc error: code = NotFound desc = could not find container \"d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7\": container with ID starting with d2bb868cad38ca38f3d260719480877aeac078df984acd7e118a290774e58fd7 not found: ID does not exist"
Sep 07 11:32:26.946401 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.946358    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9d9b666fc-2bvgl" podStartSLOduration=2.519991378 podStartE2EDuration="2.946343111s" totalImagesPullingTime="426.351733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:32:26.945507169 +0000 UTC m=+578.892857952" watchObservedRunningTime="2026-09-07 11:32:26.946343111 +0000 UTC m=+578.893693896"
Sep 07 11:32:26.969309 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.969283    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-wqp98"]
Sep 07 11:32:26.974324 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.974303    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-wqp98"]
Sep 07 11:32:26.979034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.979005    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gnk8c"]
Sep 07 11:32:26.979175 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:26.979153    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-gnk8c" podUID="c9207675-217c-460a-b147-7d145e9ff1fe" containerName="authorino" containerID="cri-o://6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46" gracePeriod=30
Sep 07 11:32:27.217862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.217836    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:32:27.296482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.296451    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6h8g\" (UniqueName: \"kubernetes.io/projected/c9207675-217c-460a-b147-7d145e9ff1fe-kube-api-access-k6h8g\") pod \"c9207675-217c-460a-b147-7d145e9ff1fe\" (UID: \"c9207675-217c-460a-b147-7d145e9ff1fe\") "
Sep 07 11:32:27.298470 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.298446    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9207675-217c-460a-b147-7d145e9ff1fe-kube-api-access-k6h8g" (OuterVolumeSpecName: "kube-api-access-k6h8g") pod "c9207675-217c-460a-b147-7d145e9ff1fe" (UID: "c9207675-217c-460a-b147-7d145e9ff1fe"). InnerVolumeSpecName "kube-api-access-k6h8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:32:27.397687 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.397645    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6h8g\" (UniqueName: \"kubernetes.io/projected/c9207675-217c-460a-b147-7d145e9ff1fe-kube-api-access-k6h8g\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:27.934621 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.934586    2560 generic.go:356] "Generic (PLEG): container finished" podID="c9207675-217c-460a-b147-7d145e9ff1fe" containerID="6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46" exitCode=0
Sep 07 11:32:27.935144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.934640    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gnk8c"
Sep 07 11:32:27.935144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.934680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gnk8c" event={"ID":"c9207675-217c-460a-b147-7d145e9ff1fe","Type":"ContainerDied","Data":"6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46"}
Sep 07 11:32:27.935144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.934721    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gnk8c" event={"ID":"c9207675-217c-460a-b147-7d145e9ff1fe","Type":"ContainerDied","Data":"c25c7d2a6b1b5ef0e7c6043e70996be45e8fe8221ecdd9492aa74815de83c702"}
Sep 07 11:32:27.935144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.934759    2560 scope.go:117] "RemoveContainer" containerID="6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46"
Sep 07 11:32:27.942798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.942782    2560 scope.go:117] "RemoveContainer" containerID="6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46"
Sep 07 11:32:27.943073 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:27.943049    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46\": container with ID starting with 6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46 not found: ID does not exist" containerID="6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46"
Sep 07 11:32:27.943134 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.943081    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46"} err="failed to get container status \"6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46\": rpc error: code = NotFound desc = could not find container \"6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46\": container with ID starting with 6e281569be154ef81a0044f19f8b9ba01db312d07692ff6c81c3f241eceaca46 not found: ID does not exist"
Sep 07 11:32:27.963162 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.963140    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gnk8c"]
Sep 07 11:32:27.969612 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:27.969595    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gnk8c"]
Sep 07 11:32:28.497313 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:28.497277    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c7fe30-087b-4421-897e-32a9e2e1c7bc" path="/var/lib/kubelet/pods/48c7fe30-087b-4421-897e-32a9e2e1c7bc/volumes"
Sep 07 11:32:28.497635 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:28.497618    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9207675-217c-460a-b147-7d145e9ff1fe" path="/var/lib/kubelet/pods/c9207675-217c-460a-b147-7d145e9ff1fe/volumes"
Sep 07 11:32:32.130608 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.130578    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7746ccd9d4-4w475"]
Sep 07 11:32:32.131069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.130940    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9207675-217c-460a-b147-7d145e9ff1fe" containerName="authorino"
Sep 07 11:32:32.131069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.130953    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9207675-217c-460a-b147-7d145e9ff1fe" containerName="authorino"
Sep 07 11:32:32.131069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.130960    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c7fe30-087b-4421-897e-32a9e2e1c7bc" containerName="authorino"
Sep 07 11:32:32.131069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.130975    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c7fe30-087b-4421-897e-32a9e2e1c7bc" containerName="authorino"
Sep 07 11:32:32.131069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.131032    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c7fe30-087b-4421-897e-32a9e2e1c7bc" containerName="authorino"
Sep 07 11:32:32.131069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.131042    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9207675-217c-460a-b147-7d145e9ff1fe" containerName="authorino"
Sep 07 11:32:32.154675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.154637    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7746ccd9d4-4w475"]
Sep 07 11:32:32.154848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.154685    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.157607 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.157572    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4pz7p\""
Sep 07 11:32:32.157926 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.157908    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 11:32:32.158021 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.157906    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 11:32:32.158215 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.158201    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 11:32:32.242481 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.242446    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-webhook-cert\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.242678 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.242494    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.242678 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.242566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4gsf\" (UniqueName: \"kubernetes.io/projected/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-kube-api-access-k4gsf\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.343761 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.343714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-webhook-cert\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.343945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.343783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.343945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.343813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4gsf\" (UniqueName: \"kubernetes.io/projected/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-kube-api-access-k4gsf\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.343945 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:32.343936    2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 07 11:32:32.344114 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:32.344012    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs podName:3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383 nodeName:}" failed. No retries permitted until 2026-09-07 11:32:32.843994746 +0000 UTC m=+584.791345514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs") pod "maas-controller-7746ccd9d4-4w475" (UID: "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383") : secret "maas-controller-metrics-tls" not found
Sep 07 11:32:32.346108 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.346088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-webhook-cert\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.360403 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.360369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4gsf\" (UniqueName: \"kubernetes.io/projected/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-kube-api-access-k4gsf\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.848683 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.848638    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:32.850957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:32.850938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs\") pod \"maas-controller-7746ccd9d4-4w475\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") " pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:33.066666 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:33.066633    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:33.339583 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:33.339560    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7746ccd9d4-4w475"]
Sep 07 11:32:33.342185 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:32:33.342155    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd4a8dc_1b48_47e3_bb4f_9fd5f7d6f383.slice/crio-bac7a6bd0965e3da68730b6631716b3467834e2fd33918630f5f925f3f67ff6c WatchSource:0}: Error finding container bac7a6bd0965e3da68730b6631716b3467834e2fd33918630f5f925f3f67ff6c: Status 404 returned error can't find the container with id bac7a6bd0965e3da68730b6631716b3467834e2fd33918630f5f925f3f67ff6c
Sep 07 11:32:33.965258 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:33.965171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7746ccd9d4-4w475" event={"ID":"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383","Type":"ContainerStarted","Data":"bac7a6bd0965e3da68730b6631716b3467834e2fd33918630f5f925f3f67ff6c"}
Sep 07 11:32:35.982614 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:35.982580    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7746ccd9d4-4w475" event={"ID":"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383","Type":"ContainerStarted","Data":"fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2"}
Sep 07 11:32:35.983024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:35.982726    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:36.065702 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:36.065634    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7746ccd9d4-4w475" podStartSLOduration=1.889830667 podStartE2EDuration="4.065614757s" totalImagesPullingTime="2.17578409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:32:36.064519109 +0000 UTC m=+588.011869895" watchObservedRunningTime="2026-09-07 11:32:36.065614757 +0000 UTC m=+588.012965542"
Sep 07 11:32:46.364771 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.364722    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7746ccd9d4-4w475"]
Sep 07 11:32:46.365202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.364967    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7746ccd9d4-4w475" podUID="3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" containerName="manager" containerID="cri-o://fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2" gracePeriod=30
Sep 07 11:32:46.366862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.366836    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:46.612129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.612103    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:46.672872 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.672784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-webhook-cert\") pod \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") "
Sep 07 11:32:46.672872 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.672829    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs\") pod \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") "
Sep 07 11:32:46.672872 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.672866    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4gsf\" (UniqueName: \"kubernetes.io/projected/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-kube-api-access-k4gsf\") pod \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\" (UID: \"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383\") "
Sep 07 11:32:46.675084 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.675050    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-kube-api-access-k4gsf" (OuterVolumeSpecName: "kube-api-access-k4gsf") pod "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" (UID: "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383"). InnerVolumeSpecName "kube-api-access-k4gsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:32:46.675187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.675094    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" (UID: "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:32:46.675187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.675135    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" (UID: "3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:32:46.773860 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.773823    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4gsf\" (UniqueName: \"kubernetes.io/projected/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-kube-api-access-k4gsf\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:46.773860 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.773854    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-webhook-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:46.773860 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:46.773868    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383-metrics-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:32:47.034005 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.033960    2560 generic.go:356] "Generic (PLEG): container finished" podID="3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" containerID="fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2" exitCode=0
Sep 07 11:32:47.034174 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.034016    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7746ccd9d4-4w475"
Sep 07 11:32:47.034174 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.034046    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7746ccd9d4-4w475" event={"ID":"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383","Type":"ContainerDied","Data":"fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2"}
Sep 07 11:32:47.034174 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.034117    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7746ccd9d4-4w475" event={"ID":"3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383","Type":"ContainerDied","Data":"bac7a6bd0965e3da68730b6631716b3467834e2fd33918630f5f925f3f67ff6c"}
Sep 07 11:32:47.034174 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.034141    2560 scope.go:117] "RemoveContainer" containerID="fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2"
Sep 07 11:32:47.042986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.042958    2560 scope.go:117] "RemoveContainer" containerID="fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2"
Sep 07 11:32:47.043248 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:32:47.043229    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2\": container with ID starting with fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2 not found: ID does not exist" containerID="fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2"
Sep 07 11:32:47.043300 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.043256    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2"} err="failed to get container status \"fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2\": rpc error: code = NotFound desc = could not find container \"fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2\": container with ID starting with fd9844924be5dfebd4694ed48e351b3adfcb6d2c012837da4c9aad5bc42aaee2 not found: ID does not exist"
Sep 07 11:32:47.057798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.057772    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7746ccd9d4-4w475"]
Sep 07 11:32:47.060224 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:47.060201    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7746ccd9d4-4w475"]
Sep 07 11:32:48.480431 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:48.480400    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:32:48.480890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:48.480871    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:32:48.496625 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:32:48.496599    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" path="/var/lib/kubelet/pods/3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383/volumes"
Sep 07 11:33:09.566126 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.566089    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"]
Sep 07 11:33:09.568081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.566438    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" containerName="manager"
Sep 07 11:33:09.568081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.566449    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" containerName="manager"
Sep 07 11:33:09.568081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.566518    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd4a8dc-1b48-47e3-bb4f-9fd5f7d6f383" containerName="manager"
Sep 07 11:33:09.569040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.569024    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.572420 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.572393    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-659nh\""
Sep 07 11:33:09.574340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.574320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 07 11:33:09.590620 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.590591    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"]
Sep 07 11:33:09.677138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.677103    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d9a6776-8a2f-4f40-94b3-3b3c8f79de22-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hjfht\" (UID: \"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.677339 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.677179    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cshj6\" (UniqueName: \"kubernetes.io/projected/8d9a6776-8a2f-4f40-94b3-3b3c8f79de22-kube-api-access-cshj6\") pod \"payload-pre-processing-59fbc47bf8-hjfht\" (UID: \"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.698134 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.698092    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"]
Sep 07 11:33:09.701302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.701282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:09.712613 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.712585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"]
Sep 07 11:33:09.778367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.778330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cshj6\" (UniqueName: \"kubernetes.io/projected/8d9a6776-8a2f-4f40-94b3-3b3c8f79de22-kube-api-access-cshj6\") pod \"payload-pre-processing-59fbc47bf8-hjfht\" (UID: \"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.778566 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.778434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d9a6776-8a2f-4f40-94b3-3b3c8f79de22-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hjfht\" (UID: \"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.779145 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.779124    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d9a6776-8a2f-4f40-94b3-3b3c8f79de22-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hjfht\" (UID: \"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.801204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.801168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cshj6\" (UniqueName: \"kubernetes.io/projected/8d9a6776-8a2f-4f40-94b3-3b3c8f79de22-kube-api-access-cshj6\") pod \"payload-pre-processing-59fbc47bf8-hjfht\" (UID: \"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.879143 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.879057    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:09.879299 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.879275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc264ca-820b-46cc-974f-e6feceb75d72-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pm4dl\" (UID: \"bbc264ca-820b-46cc-974f-e6feceb75d72\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:09.879440 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.879410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swdx9\" (UniqueName: \"kubernetes.io/projected/bbc264ca-820b-46cc-974f-e6feceb75d72-kube-api-access-swdx9\") pod \"payload-processing-5b54b7fbb7-pm4dl\" (UID: \"bbc264ca-820b-46cc-974f-e6feceb75d72\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:09.980356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.980315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swdx9\" (UniqueName: \"kubernetes.io/projected/bbc264ca-820b-46cc-974f-e6feceb75d72-kube-api-access-swdx9\") pod \"payload-processing-5b54b7fbb7-pm4dl\" (UID: \"bbc264ca-820b-46cc-974f-e6feceb75d72\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:09.980541 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.980383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc264ca-820b-46cc-974f-e6feceb75d72-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pm4dl\" (UID: \"bbc264ca-820b-46cc-974f-e6feceb75d72\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:09.981282 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.981030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc264ca-820b-46cc-974f-e6feceb75d72-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pm4dl\" (UID: \"bbc264ca-820b-46cc-974f-e6feceb75d72\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:09.992819 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:09.992785    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swdx9\" (UniqueName: \"kubernetes.io/projected/bbc264ca-820b-46cc-974f-e6feceb75d72-kube-api-access-swdx9\") pod \"payload-processing-5b54b7fbb7-pm4dl\" (UID: \"bbc264ca-820b-46cc-974f-e6feceb75d72\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:10.013458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:10.013425    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:10.178960 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:10.178934    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"]
Sep 07 11:33:10.181226 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:33:10.181200    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9a6776_8a2f_4f40_94b3_3b3c8f79de22.slice/crio-f388c1afd015cc85bb61a6572e3eb48c115d9dfc8b0b4e35210fc35759f5d91f WatchSource:0}: Error finding container f388c1afd015cc85bb61a6572e3eb48c115d9dfc8b0b4e35210fc35759f5d91f: Status 404 returned error can't find the container with id f388c1afd015cc85bb61a6572e3eb48c115d9dfc8b0b4e35210fc35759f5d91f
Sep 07 11:33:10.183973 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:10.183954    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:33:10.291845 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:10.291805    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"]
Sep 07 11:33:10.298520 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:33:10.298478    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc264ca_820b_46cc_974f_e6feceb75d72.slice/crio-58d85771286c5cc036e81982dbab2b9fe73a37c181f57013b904541561ea7c6c WatchSource:0}: Error finding container 58d85771286c5cc036e81982dbab2b9fe73a37c181f57013b904541561ea7c6c: Status 404 returned error can't find the container with id 58d85771286c5cc036e81982dbab2b9fe73a37c181f57013b904541561ea7c6c
Sep 07 11:33:11.146854 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:11.146753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht" event={"ID":"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22","Type":"ContainerStarted","Data":"f388c1afd015cc85bb61a6572e3eb48c115d9dfc8b0b4e35210fc35759f5d91f"}
Sep 07 11:33:11.148505 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:11.148475    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl" event={"ID":"bbc264ca-820b-46cc-974f-e6feceb75d72","Type":"ContainerStarted","Data":"58d85771286c5cc036e81982dbab2b9fe73a37c181f57013b904541561ea7c6c"}
Sep 07 11:33:13.160727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:13.160640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht" event={"ID":"8d9a6776-8a2f-4f40-94b3-3b3c8f79de22","Type":"ContainerStarted","Data":"99b1324998285ca5d5c714637cb8739276352d4537e20d8fa94fc3fa0931c8d3"}
Sep 07 11:33:13.160727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:13.160707    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:13.164710 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:13.164684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl" event={"ID":"bbc264ca-820b-46cc-974f-e6feceb75d72","Type":"ContainerStarted","Data":"8ebdc1b965ab920c3bb874cb471a7d92a89c6f780e90cfa097227d99ddd07c85"}
Sep 07 11:33:13.164796 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:13.164763    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:13.178014 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:13.177967    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht" podStartSLOduration=1.857431914 podStartE2EDuration="4.177952091s" totalImagesPullingTime="2.320520177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:33:13.177086106 +0000 UTC m=+625.124436890" watchObservedRunningTime="2026-09-07 11:33:13.177952091 +0000 UTC m=+625.125302875"
Sep 07 11:33:13.194418 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:13.192082    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl" podStartSLOduration=1.971004909 podStartE2EDuration="4.192061744s" totalImagesPullingTime="2.221056835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:33:13.191619018 +0000 UTC m=+625.138969804" watchObservedRunningTime="2026-09-07 11:33:13.192061744 +0000 UTC m=+625.139412529"
Sep 07 11:33:19.196275 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:19.196184    2560 generic.go:356] "Generic (PLEG): container finished" podID="92a89314-0821-4934-a99c-74b3a9f3e3eb" containerID="fc93500336e4b3adbe6105309d287746f49145a6e9b6c6598f90696e6869ff18" exitCode=0
Sep 07 11:33:19.196275 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:19.196257    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" event={"ID":"92a89314-0821-4934-a99c-74b3a9f3e3eb","Type":"ContainerDied","Data":"fc93500336e4b3adbe6105309d287746f49145a6e9b6c6598f90696e6869ff18"}
Sep 07 11:33:19.196689 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:19.196643    2560 scope.go:117] "RemoveContainer" containerID="fc93500336e4b3adbe6105309d287746f49145a6e9b6c6598f90696e6869ff18"
Sep 07 11:33:19.197701 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:19.197663    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:33:19.197834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:19.197783    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:33:19.197905 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:19.197842    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:33:20.207071 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:20.207032    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" event={"ID":"92a89314-0821-4934-a99c-74b3a9f3e3eb","Type":"ContainerStarted","Data":"4a821fec95853ab62ebcc63b5e73a0469cf72be782d2d16e487a563bdd971506"}
Sep 07 11:33:21.113552 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:21.113506    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:33:21.114843 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:21.114795    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qnwdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 07 11:33:21.114969 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:21.114866    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" podUID="92a89314-0821-4934-a99c-74b3a9f3e3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 07 11:33:22.113586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:22.113543    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qnwdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 07 11:33:22.114091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:22.113838    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" podUID="92a89314-0821-4934-a99c-74b3a9f3e3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 07 11:33:23.113323 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:23.113275    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qnwdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 07 11:33:23.113516 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:23.113402    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" podUID="92a89314-0821-4934-a99c-74b3a9f3e3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 07 11:33:24.113577 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:24.113507    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qnwdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 07 11:33:24.114039 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:24.113649    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb" podUID="92a89314-0821-4934-a99c-74b3a9f3e3eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 07 11:33:24.172944 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:24.172908    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-pm4dl"
Sep 07 11:33:24.173252 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:24.173234    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hjfht"
Sep 07 11:33:24.682234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:24.682200    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:33:25.146214 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:25.146182    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:33:25.234775 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:25.234730    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qnwdb"
Sep 07 11:33:39.099697 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.099656    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"]
Sep 07 11:33:39.102453 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.102437    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.104786 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.104732    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 11:33:39.105392 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.105370    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 11:33:39.105507 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.105487    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bf2b6\""
Sep 07 11:33:39.105562 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.105537    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 07 11:33:39.115793 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.115769    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"]
Sep 07 11:33:39.175954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.175920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqm8j\" (UniqueName: \"kubernetes.io/projected/a2d451f4-150b-4384-bdbb-897538778d55-kube-api-access-bqm8j\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.176118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.175965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.176118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.176051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.176118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.176110    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.176219 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.176158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d451f4-150b-4384-bdbb-897538778d55-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.176219 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.176186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.276667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.276632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqm8j\" (UniqueName: \"kubernetes.io/projected/a2d451f4-150b-4384-bdbb-897538778d55-kube-api-access-bqm8j\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.276667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.276676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.276948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.276705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.276948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.276729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.276948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.276767    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d451f4-150b-4384-bdbb-897538778d55-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.276948 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.276795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.277207 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.277184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.277266 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.277208    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.277266 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.277246    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.279111 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.279088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2d451f4-150b-4384-bdbb-897538778d55-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.279265 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.279247    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d451f4-150b-4384-bdbb-897538778d55-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.284833 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.284811    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqm8j\" (UniqueName: \"kubernetes.io/projected/a2d451f4-150b-4384-bdbb-897538778d55-kube-api-access-bqm8j\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6\" (UID: \"a2d451f4-150b-4384-bdbb-897538778d55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.413243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.413155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:39.685574 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:39.685520    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"]
Sep 07 11:33:39.688015 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:33:39.687982    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d451f4_150b_4384_bdbb_897538778d55.slice/crio-7030881b4637dc70da6d7f5a8a480cbcce1298a111c9e374c58488713f6ddd72 WatchSource:0}: Error finding container 7030881b4637dc70da6d7f5a8a480cbcce1298a111c9e374c58488713f6ddd72: Status 404 returned error can't find the container with id 7030881b4637dc70da6d7f5a8a480cbcce1298a111c9e374c58488713f6ddd72
Sep 07 11:33:40.308234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:40.308178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6" event={"ID":"a2d451f4-150b-4384-bdbb-897538778d55","Type":"ContainerStarted","Data":"7030881b4637dc70da6d7f5a8a480cbcce1298a111c9e374c58488713f6ddd72"}
Sep 07 11:33:41.314344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:41.314304    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6" event={"ID":"a2d451f4-150b-4384-bdbb-897538778d55","Type":"ContainerStarted","Data":"0dcdf401026070248c761a6e8b149fb68c5cabff86fc8b4ac60a736e656319fe"}
Sep 07 11:33:41.334248 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:41.334194    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6" podStartSLOduration=1.247612969 podStartE2EDuration="2.334176798s" totalImagesPullingTime="1.086563829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:33:41.332386543 +0000 UTC m=+653.279737328" watchObservedRunningTime="2026-09-07 11:33:41.334176798 +0000 UTC m=+653.281527581"
Sep 07 11:33:49.413902 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:49.413858    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:49.414314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:49.413918    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:49.426321 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:49.426282    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:33:50.368109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:33:50.368081    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6"
Sep 07 11:34:03.002388 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.002353    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"]
Sep 07 11:34:03.005651 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.005633    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.007692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.007666    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 07 11:34:03.014556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.014532    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"]
Sep 07 11:34:03.195523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.195489    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.195523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.195537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.195857 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.195563    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.195857 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.195632    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.195857 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.195692    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.195857 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.195720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8mzx\" (UniqueName: \"kubernetes.io/projected/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-kube-api-access-g8mzx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.296618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.296545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.296618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.296576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.296618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.296595    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.296618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.296620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.296967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.296665    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.296967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.296694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8mzx\" (UniqueName: \"kubernetes.io/projected/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-kube-api-access-g8mzx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.297133 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.297100    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.297259 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.297112    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.297259 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.297237    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.298953 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.298927    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.299256 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.299234    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.304617 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.304594    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8mzx\" (UniqueName: \"kubernetes.io/projected/ed4b4513-6197-4fa6-a2f7-efe00fbc75a9-kube-api-access-g8mzx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6rgp4\" (UID: \"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.315380 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.315357    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:03.585752 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:03.585688    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"]
Sep 07 11:34:03.588117 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:34:03.588094    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4b4513_6197_4fa6_a2f7_efe00fbc75a9.slice/crio-9ded025882eda292980f7eb2510a9b684685b6ec1af0cb11919be8241ee24735 WatchSource:0}: Error finding container 9ded025882eda292980f7eb2510a9b684685b6ec1af0cb11919be8241ee24735: Status 404 returned error can't find the container with id 9ded025882eda292980f7eb2510a9b684685b6ec1af0cb11919be8241ee24735
Sep 07 11:34:04.424830 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:04.424790    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4" event={"ID":"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9","Type":"ContainerStarted","Data":"c08e1cfd63f47eca587f40c0686bd4660e3bbe995fbd3ed356ed4e9f3ab7416c"}
Sep 07 11:34:04.425258 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:04.424856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4" event={"ID":"ed4b4513-6197-4fa6-a2f7-efe00fbc75a9","Type":"ContainerStarted","Data":"9ded025882eda292980f7eb2510a9b684685b6ec1af0cb11919be8241ee24735"}
Sep 07 11:34:04.442965 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:04.442911    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4" podStartSLOduration=2.249471883 podStartE2EDuration="2.442893997s" totalImagesPullingTime="193.422114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:34:04.441472891 +0000 UTC m=+676.388823677" watchObservedRunningTime="2026-09-07 11:34:04.442893997 +0000 UTC m=+676.390244781"
Sep 07 11:34:13.316457 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:13.316409    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:13.316940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:13.316474    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:13.329016 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:13.328987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:13.477812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:13.477779    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6rgp4"
Sep 07 11:34:31.491260 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.491225    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"]
Sep 07 11:34:31.495339 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.495315    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.497548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.497522    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 07 11:34:31.505254 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.505234    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"]
Sep 07 11:34:31.631119 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.631087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.631285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.631127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.631285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.631162    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4dm8\" (UniqueName: \"kubernetes.io/projected/1a091f58-26cb-42cb-8920-671613c6b0e3-kube-api-access-r4dm8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.631285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.631215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.631285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.631275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a091f58-26cb-42cb-8920-671613c6b0e3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.631423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.631305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.731817 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.731780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.731833    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4dm8\" (UniqueName: \"kubernetes.io/projected/1a091f58-26cb-42cb-8920-671613c6b0e3-kube-api-access-r4dm8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.731854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.731879    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a091f58-26cb-42cb-8920-671613c6b0e3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.731898    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.731964    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732274 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.732201    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732274 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.732254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.732384 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.732301    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.734155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.734124    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a091f58-26cb-42cb-8920-671613c6b0e3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.734461 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.734440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a091f58-26cb-42cb-8920-671613c6b0e3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.741157 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.741135    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4dm8\" (UniqueName: \"kubernetes.io/projected/1a091f58-26cb-42cb-8920-671613c6b0e3-kube-api-access-r4dm8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ljndv\" (UID: \"1a091f58-26cb-42cb-8920-671613c6b0e3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:31.807868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:31.807800    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:32.077511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:32.077433    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"]
Sep 07 11:34:32.081209 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:34:32.081176    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a091f58_26cb_42cb_8920_671613c6b0e3.slice/crio-07d23cbfd09a02ba12568226d0d20cf34fb990e227199dbffa1f389372bdead2 WatchSource:0}: Error finding container 07d23cbfd09a02ba12568226d0d20cf34fb990e227199dbffa1f389372bdead2: Status 404 returned error can't find the container with id 07d23cbfd09a02ba12568226d0d20cf34fb990e227199dbffa1f389372bdead2
Sep 07 11:34:32.563179 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:32.563118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv" event={"ID":"1a091f58-26cb-42cb-8920-671613c6b0e3","Type":"ContainerStarted","Data":"8fc345c507fc6d468e9031350d0fc77936be151b625710429c5c95e817f53083"}
Sep 07 11:34:32.563569 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:32.563262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv" event={"ID":"1a091f58-26cb-42cb-8920-671613c6b0e3","Type":"ContainerStarted","Data":"07d23cbfd09a02ba12568226d0d20cf34fb990e227199dbffa1f389372bdead2"}
Sep 07 11:34:32.580922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:32.580863    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv" podStartSLOduration=1.389099839 podStartE2EDuration="1.580847633s" totalImagesPullingTime="191.747794ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:34:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:34:32.579657994 +0000 UTC m=+704.527008783" watchObservedRunningTime="2026-09-07 11:34:32.580847633 +0000 UTC m=+704.528198416"
Sep 07 11:34:41.808910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:41.808816    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:41.808910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:41.808904    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:41.822356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:41.822329    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:34:42.625641 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:34:42.625615    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ljndv"
Sep 07 11:35:42.834235 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:42.834189    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9d9b666fc-2bvgl"]
Sep 07 11:35:42.834766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:42.834430    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-9d9b666fc-2bvgl" podUID="dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" containerName="authorino" containerID="cri-o://7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c" gracePeriod=30
Sep 07 11:35:43.076090 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.076065    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:35:43.097492 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.097409    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-tls-cert\") pod \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") "
Sep 07 11:35:43.097636 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.097538    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-kube-api-access-59sjh\") pod \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\" (UID: \"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446\") "
Sep 07 11:35:43.099983 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.099950    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-kube-api-access-59sjh" (OuterVolumeSpecName: "kube-api-access-59sjh") pod "dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" (UID: "dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446"). InnerVolumeSpecName "kube-api-access-59sjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:35:43.108510 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.108472    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" (UID: "dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:35:43.198371 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.198336    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-tls-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:35:43.198371 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.198367    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446-kube-api-access-59sjh\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:35:43.915176 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.915143    2560 generic.go:356] "Generic (PLEG): container finished" podID="dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" containerID="7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c" exitCode=0
Sep 07 11:35:43.915582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.915183    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d9b666fc-2bvgl"
Sep 07 11:35:43.915582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.915203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9b666fc-2bvgl" event={"ID":"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446","Type":"ContainerDied","Data":"7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c"}
Sep 07 11:35:43.915582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.915244    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d9b666fc-2bvgl" event={"ID":"dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446","Type":"ContainerDied","Data":"266bd865dc2952539c7aa51bb0ebab6f1e4c13cb7124a057cbb627b20d294a43"}
Sep 07 11:35:43.915582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.915260    2560 scope.go:117] "RemoveContainer" containerID="7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c"
Sep 07 11:35:43.924507 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.924488    2560 scope.go:117] "RemoveContainer" containerID="7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c"
Sep 07 11:35:43.924811 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:35:43.924791    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c\": container with ID starting with 7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c not found: ID does not exist" containerID="7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c"
Sep 07 11:35:43.924866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.924821    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c"} err="failed to get container status \"7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c\": rpc error: code = NotFound desc = could not find container \"7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c\": container with ID starting with 7ed57b1eee08da8d0260024b4d8c410f59a3a45547d7d536466e135a7e0f3d9c not found: ID does not exist"
Sep 07 11:35:43.935396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.935372    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9d9b666fc-2bvgl"]
Sep 07 11:35:43.941432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:43.941411    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9d9b666fc-2bvgl"]
Sep 07 11:35:44.498482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:35:44.498444    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" path="/var/lib/kubelet/pods/dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446/volumes"
Sep 07 11:36:01.286947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.286913    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"]
Sep 07 11:36:01.287474 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.287394    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" containerName="authorino"
Sep 07 11:36:01.287474 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.287412    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" containerName="authorino"
Sep 07 11:36:01.287637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.287621    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dadd9b5a-e5be-4a79-a84a-fdd5ed6fc446" containerName="authorino"
Sep 07 11:36:01.292319 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.292300    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.294257 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.294233    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b7e4c1-openshift-default-dockercfg-p664z\""
Sep 07 11:36:01.302429 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.302407    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"]
Sep 07 11:36:01.361061 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-certs\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361067    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-token\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-envoy\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istiod-ca-cert\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-742nf\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-kube-api-access-742nf\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361209    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-credential-socket\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361267    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-data\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-podinfo\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.361398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.361357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-socket\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.462637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istiod-ca-cert\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.462637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462645    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-742nf\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-kube-api-access-742nf\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.462905 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462695    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-credential-socket\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.462905 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462735    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-data\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.462905 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-podinfo\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463064 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462920    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-socket\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463064 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-certs\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463064 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.462994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-token\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463228 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.463072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-envoy\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463228 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.463139    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-data\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463228 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.463205    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-credential-socket\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.463340    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istiod-ca-cert\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463490 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.463471    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-certs\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.463617 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.463597    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-socket\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.465380 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.465348    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-envoy\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.465664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.465647    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-podinfo\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.471093 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.471071    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-token\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.471414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.471395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-742nf\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-kube-api-access-742nf\") pod \"e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") " pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.604099 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.604022    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:01.877421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.877395    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"]
Sep 07 11:36:01.879797 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:01.879772    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5787665_50f7_40a0_b9b7_0cbdc44ccf58.slice/crio-b04c6d9382558f4eac18ee7d4f35ffef7d81ed0d4eaca3b306e33ce5642b222a WatchSource:0}: Error finding container b04c6d9382558f4eac18ee7d4f35ffef7d81ed0d4eaca3b306e33ce5642b222a: Status 404 returned error can't find the container with id b04c6d9382558f4eac18ee7d4f35ffef7d81ed0d4eaca3b306e33ce5642b222a
Sep 07 11:36:01.882136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.882090    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:01.882207 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.882177    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:01.882243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:01.882219    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:02.002786 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:02.002734    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" event={"ID":"e5787665-50f7-40a0-b9b7-0cbdc44ccf58","Type":"ContainerStarted","Data":"20afd7226c33644e2c00e0b5dc5930bcc6267528af8a345514735e443ec4e81c"}
Sep 07 11:36:02.002953 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:02.002806    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" event={"ID":"e5787665-50f7-40a0-b9b7-0cbdc44ccf58","Type":"ContainerStarted","Data":"b04c6d9382558f4eac18ee7d4f35ffef7d81ed0d4eaca3b306e33ce5642b222a"}
Sep 07 11:36:02.023424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:02.023372    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podStartSLOduration=1.023357116 podStartE2EDuration="1.023357116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:02.021092718 +0000 UTC m=+793.968443504" watchObservedRunningTime="2026-09-07 11:36:02.023357116 +0000 UTC m=+793.970707981"
Sep 07 11:36:02.605080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:02.605041    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:02.609940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:02.609916    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:03.007409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:03.007375    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:03.008337 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:03.008316    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:05.868862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.868815    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"]
Sep 07 11:36:05.872947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.872921    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:05.875110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.875086    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b7e4c1\""
Sep 07 11:36:05.875469 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.875086    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b7e4c1-dockercfg-khtwb\""
Sep 07 11:36:05.883913 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.883892    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"]
Sep 07 11:36:05.885584 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.885561    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"]
Sep 07 11:36:05.889019 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.889002    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:05.902303 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:05.902279    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"]
Sep 07 11:36:06.010955 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.010915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsczt\" (UniqueName: \"kubernetes.io/projected/a66910d5-fbae-4c1f-bac3-eca690e3fdec-kube-api-access-tsczt\") pod \"payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.011158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.010989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.011158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.011077    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66910d5-fbae-4c1f-bac3-eca690e3fdec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.011158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.011109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcblp\" (UniqueName: \"kubernetes.io/projected/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-kube-api-access-fcblp\") pod \"payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.112071 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.112030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsczt\" (UniqueName: \"kubernetes.io/projected/a66910d5-fbae-4c1f-bac3-eca690e3fdec-kube-api-access-tsczt\") pod \"payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.112274 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.112120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.112274 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.112190    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66910d5-fbae-4c1f-bac3-eca690e3fdec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.112408 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.112328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcblp\" (UniqueName: \"kubernetes.io/projected/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-kube-api-access-fcblp\") pod \"payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.113000 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.112976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.113132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.113078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66910d5-fbae-4c1f-bac3-eca690e3fdec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.121229 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.121166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcblp\" (UniqueName: \"kubernetes.io/projected/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-kube-api-access-fcblp\") pod \"payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.121409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.121382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsczt\" (UniqueName: \"kubernetes.io/projected/a66910d5-fbae-4c1f-bac3-eca690e3fdec-kube-api-access-tsczt\") pod \"payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.188352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.188314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:06.205912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.205676    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:06.486264 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.486240    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"]
Sep 07 11:36:06.487961 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:06.487938    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a82662b_3dca_4adb_9e6d_2777a4e2d49b.slice/crio-dd07bd5edff9a7b55c6fd39cfbabcbf3b422446546da6ad1e8282394d89f3172 WatchSource:0}: Error finding container dd07bd5edff9a7b55c6fd39cfbabcbf3b422446546da6ad1e8282394d89f3172: Status 404 returned error can't find the container with id dd07bd5edff9a7b55c6fd39cfbabcbf3b422446546da6ad1e8282394d89f3172
Sep 07 11:36:06.502702 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:06.502669    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"]
Sep 07 11:36:06.505265 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:06.505245    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66910d5_fbae_4c1f_bac3_eca690e3fdec.slice/crio-e143c28c8aa9188fc09ed060fe1c8b4e58f146c9be71234917fa508511be6451 WatchSource:0}: Error finding container e143c28c8aa9188fc09ed060fe1c8b4e58f146c9be71234917fa508511be6451: Status 404 returned error can't find the container with id e143c28c8aa9188fc09ed060fe1c8b4e58f146c9be71234917fa508511be6451
Sep 07 11:36:07.031107 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.031044    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq" event={"ID":"a66910d5-fbae-4c1f-bac3-eca690e3fdec","Type":"ContainerStarted","Data":"0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f"}
Sep 07 11:36:07.031574 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.031125    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:07.031574 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.031153    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq" event={"ID":"a66910d5-fbae-4c1f-bac3-eca690e3fdec","Type":"ContainerStarted","Data":"e143c28c8aa9188fc09ed060fe1c8b4e58f146c9be71234917fa508511be6451"}
Sep 07 11:36:07.032807 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.032783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw" event={"ID":"7a82662b-3dca-4adb-9e6d-2777a4e2d49b","Type":"ContainerStarted","Data":"66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4"}
Sep 07 11:36:07.032954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.032824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw" event={"ID":"7a82662b-3dca-4adb-9e6d-2777a4e2d49b","Type":"ContainerStarted","Data":"dd07bd5edff9a7b55c6fd39cfbabcbf3b422446546da6ad1e8282394d89f3172"}
Sep 07 11:36:07.032954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.032880    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:07.048889 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.048845    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq" podStartSLOduration=2.04883139 podStartE2EDuration="2.04883139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:07.046910074 +0000 UTC m=+798.994260872" watchObservedRunningTime="2026-09-07 11:36:07.04883139 +0000 UTC m=+798.996182173"
Sep 07 11:36:07.066154 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:07.066108    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw" podStartSLOduration=2.066094483 podStartE2EDuration="2.066094483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:07.064049834 +0000 UTC m=+799.011400620" watchObservedRunningTime="2026-09-07 11:36:07.066094483 +0000 UTC m=+799.013445267"
Sep 07 11:36:09.592501 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.592462    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"]
Sep 07 11:36:09.596303 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.596283    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.598497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.598473    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c30b7add-openshift-default-dockercfg-grz52\""
Sep 07 11:36:09.607671 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.607650    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"]
Sep 07 11:36:09.747425 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747391    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-envoy\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da065508-f26b-4431-9753-7a05d9ab1b47-istio-podinfo\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747458    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-credential-socket\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747567    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-certs\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747611    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-data\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747872 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747708    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-istio-token\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747872 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747836    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da065508-f26b-4431-9753-7a05d9ab1b47-istiod-ca-cert\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747890    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-socket\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.747987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.747920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9grx\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-kube-api-access-c9grx\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848631 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-certs\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848631 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-data\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-istio-token\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da065508-f26b-4431-9753-7a05d9ab1b47-istiod-ca-cert\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848719    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-socket\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848756    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9grx\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-kube-api-access-c9grx\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848820    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-envoy\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.848890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848859    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da065508-f26b-4431-9753-7a05d9ab1b47-istio-podinfo\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.849256 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.848892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-credential-socket\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.849256 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.849081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-socket\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.849256 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.849112    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-certs\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.849404 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.849258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-data\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.849404 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.849345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-credential-socket\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.849648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.849619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da065508-f26b-4431-9753-7a05d9ab1b47-istiod-ca-cert\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.851571 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.851501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-envoy\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.851772 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.851729    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da065508-f26b-4431-9753-7a05d9ab1b47-istio-podinfo\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.859153 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.859129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-istio-token\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.859266 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.859249    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9grx\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-kube-api-access-c9grx\") pod \"e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") " pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:09.909095 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:09.909058    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:10.187782 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:10.187673    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"]
Sep 07 11:36:10.191788 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:10.191757    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda065508_f26b_4431_9753_7a05d9ab1b47.slice/crio-7d3aca1e31d1622218101b49643406756a49f5db88550bff02332013fd3c5630 WatchSource:0}: Error finding container 7d3aca1e31d1622218101b49643406756a49f5db88550bff02332013fd3c5630: Status 404 returned error can't find the container with id 7d3aca1e31d1622218101b49643406756a49f5db88550bff02332013fd3c5630
Sep 07 11:36:10.194121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:10.194087    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:10.194205 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:10.194155    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:10.194205 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:10.194185    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:11.054661 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:11.054582    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" event={"ID":"da065508-f26b-4431-9753-7a05d9ab1b47","Type":"ContainerStarted","Data":"67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"}
Sep 07 11:36:11.054661 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:11.054636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" event={"ID":"da065508-f26b-4431-9753-7a05d9ab1b47","Type":"ContainerStarted","Data":"7d3aca1e31d1622218101b49643406756a49f5db88550bff02332013fd3c5630"}
Sep 07 11:36:11.075711 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:11.075665    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podStartSLOduration=2.075650246 podStartE2EDuration="2.075650246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:11.074645964 +0000 UTC m=+803.021996758" watchObservedRunningTime="2026-09-07 11:36:11.075650246 +0000 UTC m=+803.023001029"
Sep 07 11:36:11.910406 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:11.909818    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:11.926402 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:11.926369    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:12.060091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:12.060059    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:12.060949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:12.060930    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:14.369353 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.369323    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"]
Sep 07 11:36:14.372273 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.372258    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.374592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.374573    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c30b7add\""
Sep 07 11:36:14.375120 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.375107    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c30b7add-dockercfg-vwk5z\""
Sep 07 11:36:14.383808 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.383788    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"]
Sep 07 11:36:14.518411 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.518368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztpm\" (UniqueName: \"kubernetes.io/projected/973aecfc-c5f7-48d8-8efc-1fcbb113364d-kube-api-access-6ztpm\") pod \"payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") " pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.518602 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.518502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973aecfc-c5f7-48d8-8efc-1fcbb113364d-plugins-config-volume\") pod \"payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") " pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.619335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.619292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973aecfc-c5f7-48d8-8efc-1fcbb113364d-plugins-config-volume\") pod \"payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") " pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.619539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.619383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztpm\" (UniqueName: \"kubernetes.io/projected/973aecfc-c5f7-48d8-8efc-1fcbb113364d-kube-api-access-6ztpm\") pod \"payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") " pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.620139 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.620115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973aecfc-c5f7-48d8-8efc-1fcbb113364d-plugins-config-volume\") pod \"payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") " pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.627322 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.627292    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztpm\" (UniqueName: \"kubernetes.io/projected/973aecfc-c5f7-48d8-8efc-1fcbb113364d-kube-api-access-6ztpm\") pod \"payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") " pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.683731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.683697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:14.968947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:14.968921    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"]
Sep 07 11:36:14.970805 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:14.970778    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973aecfc_c5f7_48d8_8efc_1fcbb113364d.slice/crio-7aee27a9f498f23fcf89750524b2a78ec91952ebc5b9f7c6e1af59f6787ca052 WatchSource:0}: Error finding container 7aee27a9f498f23fcf89750524b2a78ec91952ebc5b9f7c6e1af59f6787ca052: Status 404 returned error can't find the container with id 7aee27a9f498f23fcf89750524b2a78ec91952ebc5b9f7c6e1af59f6787ca052
Sep 07 11:36:15.077028 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:15.076991    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk" event={"ID":"973aecfc-c5f7-48d8-8efc-1fcbb113364d","Type":"ContainerStarted","Data":"8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea"}
Sep 07 11:36:15.077182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:15.077055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk" event={"ID":"973aecfc-c5f7-48d8-8efc-1fcbb113364d","Type":"ContainerStarted","Data":"7aee27a9f498f23fcf89750524b2a78ec91952ebc5b9f7c6e1af59f6787ca052"}
Sep 07 11:36:15.077182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:15.077164    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:18.040291 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:18.040254    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:36:18.040699 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:18.040336    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:36:18.062949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:18.062894    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk" podStartSLOduration=4.062877258 podStartE2EDuration="4.062877258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:15.093914417 +0000 UTC m=+807.041265202" watchObservedRunningTime="2026-09-07 11:36:18.062877258 +0000 UTC m=+810.010228074"
Sep 07 11:36:19.047929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:19.047882    2560 patch_prober.go:28] interesting pod/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 11:36:19.048313 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:19.047978    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 11:36:20.105662 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:20.105632    2560 generic.go:356] "Generic (PLEG): container finished" podID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerID="20afd7226c33644e2c00e0b5dc5930bcc6267528af8a345514735e443ec4e81c" exitCode=0
Sep 07 11:36:20.106124 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:20.105699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" event={"ID":"e5787665-50f7-40a0-b9b7-0cbdc44ccf58","Type":"ContainerDied","Data":"20afd7226c33644e2c00e0b5dc5930bcc6267528af8a345514735e443ec4e81c"}
Sep 07 11:36:20.106186 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:20.106158    2560 scope.go:117] "RemoveContainer" containerID="20afd7226c33644e2c00e0b5dc5930bcc6267528af8a345514735e443ec4e81c"
Sep 07 11:36:20.107249 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:20.107218    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:20.107319 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:20.107305    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:20.107361 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:20.107338    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:21.115130 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:21.115099    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" event={"ID":"e5787665-50f7-40a0-b9b7-0cbdc44ccf58","Type":"ContainerStarted","Data":"9a0aa6ffd3e39f5b25240bef3babdc1d0e7d86fa19a717dbf05ffd6fee4412f4"}
Sep 07 11:36:21.605910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:21.605424    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:21.605910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:21.605557    2560 patch_prober.go:28] interesting pod/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 07 11:36:21.605910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:21.605600    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 07 11:36:22.605404 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:22.605359    2560 patch_prober.go:28] interesting pod/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 07 11:36:22.605929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:22.605445    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 07 11:36:23.605697 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:23.605188    2560 patch_prober.go:28] interesting pod/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 07 11:36:23.605697 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:23.605294    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 07 11:36:24.604884 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:24.604843    2560 patch_prober.go:28] interesting pod/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 07 11:36:24.605193 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:24.605164    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 07 11:36:25.609622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:25.609163    2560 patch_prober.go:28] interesting pod/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 07 11:36:25.609622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:25.609229    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 07 11:36:26.084400 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:26.084363    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:26.847163 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:26.847122    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:26.847763 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:26.847716    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:26.849398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:26.849379    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:36:32.186216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:32.186185    2560 generic.go:356] "Generic (PLEG): container finished" podID="da065508-f26b-4431-9753-7a05d9ab1b47" containerID="67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21" exitCode=0
Sep 07 11:36:32.186729 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:32.186268    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" event={"ID":"da065508-f26b-4431-9753-7a05d9ab1b47","Type":"ContainerDied","Data":"67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"}
Sep 07 11:36:32.186940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:32.186850    2560 scope.go:117] "RemoveContainer" containerID="67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"
Sep 07 11:36:32.188275 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:32.188223    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:32.188370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:32.188326    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:32.188414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:32.188370    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:33.219765 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.217633    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" event={"ID":"da065508-f26b-4431-9753-7a05d9ab1b47","Type":"ContainerStarted","Data":"7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733"}
Sep 07 11:36:33.676852 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.676818    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"]
Sep 07 11:36:33.679497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.679481    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.681810 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.681783    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-6580730c-kserve-self-signed-certs\""
Sep 07 11:36:33.689448 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.689413    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"]
Sep 07 11:36:33.821209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.821170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-home\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.821209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.821219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tls-certs\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.821433 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.821302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-model-cache\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.821433 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.821402    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tmp-dir\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.821511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.821434    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fq7t\" (UniqueName: \"kubernetes.io/projected/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-kube-api-access-7fq7t\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.821511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.821460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-dshm\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.913769 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.912924    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:33.913940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.913832    2560 patch_prober.go:28] interesting pod/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 11:36:33.913940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.913905    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 11:36:33.922244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tmp-dir\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fq7t\" (UniqueName: \"kubernetes.io/projected/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-kube-api-access-7fq7t\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922273    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-dshm\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-home\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tls-certs\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-model-cache\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922762    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tmp-dir\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-model-cache\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.922962 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.922899    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-home\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.924725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.924703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-dshm\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.924864 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.924819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tls-certs\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.930166 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.930113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fq7t\" (UniqueName: \"kubernetes.io/projected/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-kube-api-access-7fq7t\") pod \"e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") " pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:33.989312 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:33.989273    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:34.283833 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.283810    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"]
Sep 07 11:36:34.286192 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:34.286168    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd8b25b7_a7b8_490f_a62e_c0559ea533a8.slice/crio-c44f506946b0543d40deda3a205f8d8556f065235a077832f8acd6276d80ef42 WatchSource:0}: Error finding container c44f506946b0543d40deda3a205f8d8556f065235a077832f8acd6276d80ef42: Status 404 returned error can't find the container with id c44f506946b0543d40deda3a205f8d8556f065235a077832f8acd6276d80ef42
Sep 07 11:36:34.910140 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.910098    2560 patch_prober.go:28] interesting pod/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 11:36:34.910335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.910165    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 11:36:34.942821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.942789    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"]
Sep 07 11:36:34.945691 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.945670    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:34.947840 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.947818    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ef61c5-dockercfg-clx45\""
Sep 07 11:36:34.948017 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.947994    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ef61c5\""
Sep 07 11:36:34.957564 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:34.957541    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"]
Sep 07 11:36:35.035069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.035024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.035255 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.035083    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5vhl\" (UniqueName: \"kubernetes.io/projected/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-kube-api-access-q5vhl\") pod \"payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.136220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.136182    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.136395 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.136230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5vhl\" (UniqueName: \"kubernetes.io/projected/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-kube-api-access-q5vhl\") pod \"payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.137233 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.137212    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.144230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.144209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5vhl\" (UniqueName: \"kubernetes.io/projected/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-kube-api-access-q5vhl\") pod \"payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.228913 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.228877    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m" event={"ID":"bd8b25b7-a7b8-490f-a62e-c0559ea533a8","Type":"ContainerStarted","Data":"6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920"}
Sep 07 11:36:35.229093 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.228929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m" event={"ID":"bd8b25b7-a7b8-490f-a62e-c0559ea533a8","Type":"ContainerStarted","Data":"c44f506946b0543d40deda3a205f8d8556f065235a077832f8acd6276d80ef42"}
Sep 07 11:36:35.244358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.244311    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m" podStartSLOduration=2.2442962140000002 podStartE2EDuration="2.244296214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:35.243462474 +0000 UTC m=+827.190813255" watchObservedRunningTime="2026-09-07 11:36:35.244296214 +0000 UTC m=+827.191646997"
Sep 07 11:36:35.257588 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.257563    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:35.548587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.548524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"]
Sep 07 11:36:35.551017 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:35.550981    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3c50dc_86ea_4aa2_a0c5_4f94433a7a28.slice/crio-754ec1a0b7812edec1a715b6fa678f82099f5082300613d72ede955725e450a0 WatchSource:0}: Error finding container 754ec1a0b7812edec1a715b6fa678f82099f5082300613d72ede955725e450a0: Status 404 returned error can't find the container with id 754ec1a0b7812edec1a715b6fa678f82099f5082300613d72ede955725e450a0
Sep 07 11:36:35.910221 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.910117    2560 patch_prober.go:28] interesting pod/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 11:36:35.910221 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:35.910188    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 11:36:36.236181 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:36.236144    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7" event={"ID":"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28","Type":"ContainerStarted","Data":"31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534"}
Sep 07 11:36:36.236381 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:36.236202    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7" event={"ID":"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28","Type":"ContainerStarted","Data":"754ec1a0b7812edec1a715b6fa678f82099f5082300613d72ede955725e450a0"}
Sep 07 11:36:36.236455 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:36.236374    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:36.252720 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:36.252676    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7" podStartSLOduration=2.252663699 podStartE2EDuration="2.252663699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:36.252160983 +0000 UTC m=+828.199511768" watchObservedRunningTime="2026-09-07 11:36:36.252663699 +0000 UTC m=+828.200014483"
Sep 07 11:36:36.910109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:36.910062    2560 patch_prober.go:28] interesting pod/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 11:36:36.910592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:36.910147    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 11:36:37.910279 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:37.910231    2560 patch_prober.go:28] interesting pod/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 11:36:37.910786 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:37.910305    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 11:36:38.913884 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:38.913855    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:38.914257 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:38.914132    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:38.914790 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:38.914773    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:36:43.990283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:43.990233    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:43.990283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:43.990297    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:44.003167 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:44.003140    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:44.292937 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:44.292841    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:36:47.248441 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:47.248410    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:36:54.528998 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.528957    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"]
Sep 07 11:36:54.529550 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.529212    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk" podUID="973aecfc-c5f7-48d8-8efc-1fcbb113364d" containerName="payload-processing" containerID="cri-o://8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea" gracePeriod=30
Sep 07 11:36:54.785976 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.785904    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:54.820553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.820517    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"]
Sep 07 11:36:54.820821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.820792    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy" containerID="cri-o://7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733" gracePeriod=30
Sep 07 11:36:54.927009 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.926904    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ztpm\" (UniqueName: \"kubernetes.io/projected/973aecfc-c5f7-48d8-8efc-1fcbb113364d-kube-api-access-6ztpm\") pod \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") "
Sep 07 11:36:54.927220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.927046    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973aecfc-c5f7-48d8-8efc-1fcbb113364d-plugins-config-volume\") pod \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\" (UID: \"973aecfc-c5f7-48d8-8efc-1fcbb113364d\") "
Sep 07 11:36:54.928075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.927849    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/973aecfc-c5f7-48d8-8efc-1fcbb113364d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "973aecfc-c5f7-48d8-8efc-1fcbb113364d" (UID: "973aecfc-c5f7-48d8-8efc-1fcbb113364d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:36:54.930118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:54.930067    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/973aecfc-c5f7-48d8-8efc-1fcbb113364d-kube-api-access-6ztpm" (OuterVolumeSpecName: "kube-api-access-6ztpm") pod "973aecfc-c5f7-48d8-8efc-1fcbb113364d" (UID: "973aecfc-c5f7-48d8-8efc-1fcbb113364d"). InnerVolumeSpecName "kube-api-access-6ztpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:36:55.028970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.028935    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ztpm\" (UniqueName: \"kubernetes.io/projected/973aecfc-c5f7-48d8-8efc-1fcbb113364d-kube-api-access-6ztpm\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:36:55.028970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.028964    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973aecfc-c5f7-48d8-8efc-1fcbb113364d-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:36:55.341240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.341205    2560 generic.go:356] "Generic (PLEG): container finished" podID="973aecfc-c5f7-48d8-8efc-1fcbb113364d" containerID="8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea" exitCode=0
Sep 07 11:36:55.341430 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.341251    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"
Sep 07 11:36:55.341430 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.341287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk" event={"ID":"973aecfc-c5f7-48d8-8efc-1fcbb113364d","Type":"ContainerDied","Data":"8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea"}
Sep 07 11:36:55.341430 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.341355    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk" event={"ID":"973aecfc-c5f7-48d8-8efc-1fcbb113364d","Type":"ContainerDied","Data":"7aee27a9f498f23fcf89750524b2a78ec91952ebc5b9f7c6e1af59f6787ca052"}
Sep 07 11:36:55.341430 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.341387    2560 scope.go:117] "RemoveContainer" containerID="8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea"
Sep 07 11:36:55.350888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.350866    2560 scope.go:117] "RemoveContainer" containerID="8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea"
Sep 07 11:36:55.351154 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:36:55.351134    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea\": container with ID starting with 8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea not found: ID does not exist" containerID="8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea"
Sep 07 11:36:55.351215 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.351163    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea"} err="failed to get container status \"8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea\": rpc error: code = NotFound desc = could not find container \"8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea\": container with ID starting with 8d007f90b192bd2f77150fa182213f0eb7b8a0fe3bd328edea72fb4f534bdaea not found: ID does not exist"
Sep 07 11:36:55.363080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.363056    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"]
Sep 07 11:36:55.368816 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.368795    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c30b7add-7f4cfbb67-x4jfk"]
Sep 07 11:36:55.887358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.887323    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"]
Sep 07 11:36:55.887779 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.887765    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973aecfc-c5f7-48d8-8efc-1fcbb113364d" containerName="payload-processing"
Sep 07 11:36:55.887779 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.887779    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="973aecfc-c5f7-48d8-8efc-1fcbb113364d" containerName="payload-processing"
Sep 07 11:36:55.887866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.887840    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="973aecfc-c5f7-48d8-8efc-1fcbb113364d" containerName="payload-processing"
Sep 07 11:36:55.890936 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.890905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:55.893058 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.893039    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-25494d09-gw-openshift-default-dockercfg-rn5td\""
Sep 07 11:36:55.904727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:55.904702    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"]
Sep 07 11:36:56.039623 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.039579    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-token\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.039815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.039636    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-socket\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.039883 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.039804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c02e88-5ccd-461c-9778-0af1e485d4e7-istiod-ca-cert\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.039927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.039860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-credential-socket\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.039980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.039927    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-certs\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.039980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.039952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-envoy\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.040074 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.040011    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmdp6\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-kube-api-access-mmdp6\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.040074 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.040050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-data\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.040148 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.040082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-podinfo\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.141679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-socket\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.141877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141695    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c02e88-5ccd-461c-9778-0af1e485d4e7-istiod-ca-cert\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.141877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-credential-socket\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.141877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-certs\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.141877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-envoy\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.141877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141875    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdp6\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-kube-api-access-mmdp6\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-data\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-podinfo\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.141988    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-token\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.142141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-credential-socket\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142732 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.142621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-socket\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142732 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.142678    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c02e88-5ccd-461c-9778-0af1e485d4e7-istiod-ca-cert\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.142929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.142875    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-data\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.143231 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.143205    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-certs\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.145163 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.145136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-envoy\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.146453 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.146428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-podinfo\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.150459 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.150436    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdp6\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-kube-api-access-mmdp6\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.151122 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.151092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-token\") pod \"e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") " pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.201278 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.201240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:56.474820 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.474792    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"]
Sep 07 11:36:56.476966 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:36:56.476937    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c02e88_5ccd_461c_9778_0af1e485d4e7.slice/crio-70b03978cfb41bc75d6e2d52f89e21f8aa2ad7638066748dce56b423f01c0111 WatchSource:0}: Error finding container 70b03978cfb41bc75d6e2d52f89e21f8aa2ad7638066748dce56b423f01c0111: Status 404 returned error can't find the container with id 70b03978cfb41bc75d6e2d52f89e21f8aa2ad7638066748dce56b423f01c0111
Sep 07 11:36:56.479457 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.479422    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:56.479548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.479490    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:56.479548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.479521    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:36:56.499901 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:56.499873    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="973aecfc-c5f7-48d8-8efc-1fcbb113364d" path="/var/lib/kubelet/pods/973aecfc-c5f7-48d8-8efc-1fcbb113364d/volumes"
Sep 07 11:36:57.354452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:57.354415    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" event={"ID":"04c02e88-5ccd-461c-9778-0af1e485d4e7","Type":"ContainerStarted","Data":"02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"}
Sep 07 11:36:57.354882 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:57.354471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" event={"ID":"04c02e88-5ccd-461c-9778-0af1e485d4e7","Type":"ContainerStarted","Data":"70b03978cfb41bc75d6e2d52f89e21f8aa2ad7638066748dce56b423f01c0111"}
Sep 07 11:36:57.385417 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:57.385363    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podStartSLOduration=2.385347143 podStartE2EDuration="2.385347143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:36:57.383138731 +0000 UTC m=+849.330489518" watchObservedRunningTime="2026-09-07 11:36:57.385347143 +0000 UTC m=+849.332697927"
Sep 07 11:36:58.203340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:58.203300    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:58.209169 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:58.209138    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:58.360146 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:58.360110    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:58.361244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:58.361225    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:36:59.786499 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.786459    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"]
Sep 07 11:36:59.790251 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.790230    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:36:59.792808 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.792784    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-25494d09\""
Sep 07 11:36:59.792920 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.792813    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-25494d09-dockercfg-g76pb\""
Sep 07 11:36:59.805181 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.805156    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"]
Sep 07 11:36:59.981894 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.981846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtsmv\" (UniqueName: \"kubernetes.io/projected/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-kube-api-access-vtsmv\") pod \"payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") " pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:36:59.982075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:36:59.981945    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-plugins-config-volume\") pod \"payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") " pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:00.070427 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.070402    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:37:00.082803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.082776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtsmv\" (UniqueName: \"kubernetes.io/projected/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-kube-api-access-vtsmv\") pod \"payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") " pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:00.082896 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.082829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-plugins-config-volume\") pod \"payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") " pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:00.083424 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.083406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-plugins-config-volume\") pod \"payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") " pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:00.096777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.096734    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtsmv\" (UniqueName: \"kubernetes.io/projected/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-kube-api-access-vtsmv\") pod \"payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") " pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:00.101434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.101411    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:00.183471 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.183444    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-data\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.183632 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.183503    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da065508-f26b-4431-9753-7a05d9ab1b47-istiod-ca-cert\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.183632 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.183531    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-socket\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.184106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.183788    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9grx\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-kube-api-access-c9grx\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.184106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.183855    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-data" (OuterVolumeSpecName: "istio-data") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:00.184106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.183917    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184453    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da065508-f26b-4431-9753-7a05d9ab1b47-istio-podinfo\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184514    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-certs\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184600    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-credential-socket\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184642    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-istio-token\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184658    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da065508-f26b-4431-9753-7a05d9ab1b47-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184698    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-envoy\") pod \"da065508-f26b-4431-9753-7a05d9ab1b47\" (UID: \"da065508-f26b-4431-9753-7a05d9ab1b47\") "
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.184952    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.185097    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.185117    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da065508-f26b-4431-9753-7a05d9ab1b47-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.185130    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.185298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.185143    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.186108 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.185314    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:00.186922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.186895    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-kube-api-access-c9grx" (OuterVolumeSpecName: "kube-api-access-c9grx") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "kube-api-access-c9grx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:37:00.187366 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.187339    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:00.187447 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.187411    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-istio-token" (OuterVolumeSpecName: "istio-token") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:37:00.187628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.187604    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da065508-f26b-4431-9753-7a05d9ab1b47-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da065508-f26b-4431-9753-7a05d9ab1b47" (UID: "da065508-f26b-4431-9753-7a05d9ab1b47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:37:00.285980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.285949    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.285980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.285977    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.285980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.285987    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da065508-f26b-4431-9753-7a05d9ab1b47-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.286207 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.285996    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9grx\" (UniqueName: \"kubernetes.io/projected/da065508-f26b-4431-9753-7a05d9ab1b47-kube-api-access-c9grx\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.286207 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.286005    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da065508-f26b-4431-9753-7a05d9ab1b47-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:00.373643 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.373612    2560 generic.go:356] "Generic (PLEG): container finished" podID="da065508-f26b-4431-9753-7a05d9ab1b47" containerID="7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733" exitCode=0
Sep 07 11:37:00.373916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.373703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" event={"ID":"da065508-f26b-4431-9753-7a05d9ab1b47","Type":"ContainerDied","Data":"7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733"}
Sep 07 11:37:00.373916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.373732    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"
Sep 07 11:37:00.373916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.373776    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd" event={"ID":"da065508-f26b-4431-9753-7a05d9ab1b47","Type":"ContainerDied","Data":"7d3aca1e31d1622218101b49643406756a49f5db88550bff02332013fd3c5630"}
Sep 07 11:37:00.373916 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.373808    2560 scope.go:117] "RemoveContainer" containerID="7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733"
Sep 07 11:37:00.374609 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.374583    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"]
Sep 07 11:37:00.376261 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:37:00.376235    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f98e78a_78d5_4c9a_850c_51ec15df4fbe.slice/crio-779b02962d02a7d9c0e09c11bf6323360685db6ebc307212d3a29b4175bd6b28 WatchSource:0}: Error finding container 779b02962d02a7d9c0e09c11bf6323360685db6ebc307212d3a29b4175bd6b28: Status 404 returned error can't find the container with id 779b02962d02a7d9c0e09c11bf6323360685db6ebc307212d3a29b4175bd6b28
Sep 07 11:37:00.391247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.390896    2560 scope.go:117] "RemoveContainer" containerID="67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"
Sep 07 11:37:00.402892 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.402574    2560 scope.go:117] "RemoveContainer" containerID="7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733"
Sep 07 11:37:00.403104 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:37:00.402925    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733\": container with ID starting with 7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733 not found: ID does not exist" containerID="7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733"
Sep 07 11:37:00.403104 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.402960    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733"} err="failed to get container status \"7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733\": rpc error: code = NotFound desc = could not find container \"7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733\": container with ID starting with 7e0f98c5c19ad4888c49202a296f0ce385707f3f12056ba3187106ed07aa0733 not found: ID does not exist"
Sep 07 11:37:00.403104 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.402981    2560 scope.go:117] "RemoveContainer" containerID="67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"
Sep 07 11:37:00.404792 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:37:00.403476    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21\": container with ID starting with 67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21 not found: ID does not exist" containerID="67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"
Sep 07 11:37:00.404792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.403506    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21"} err="failed to get container status \"67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21\": rpc error: code = NotFound desc = could not find container \"67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21\": container with ID starting with 67611aa7ab51183116b52a58c9136987714e63876e1a5ec996d1162138df7a21 not found: ID does not exist"
Sep 07 11:37:00.408187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.408167    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"]
Sep 07 11:37:00.410890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.410868    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c30b7add-openshift-default-56656fcf6d-vq2fd"]
Sep 07 11:37:00.498013 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:00.497971    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" path="/var/lib/kubelet/pods/da065508-f26b-4431-9753-7a05d9ab1b47/volumes"
Sep 07 11:37:01.382048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:01.382004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6" event={"ID":"3f98e78a-78d5-4c9a-850c-51ec15df4fbe","Type":"ContainerStarted","Data":"11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4"}
Sep 07 11:37:01.382464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:01.382085    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6" event={"ID":"3f98e78a-78d5-4c9a-850c-51ec15df4fbe","Type":"ContainerStarted","Data":"779b02962d02a7d9c0e09c11bf6323360685db6ebc307212d3a29b4175bd6b28"}
Sep 07 11:37:01.382464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:01.382129    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:01.399964 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:01.399917    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6" podStartSLOduration=2.399902447 podStartE2EDuration="2.399902447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:37:01.39846213 +0000 UTC m=+853.345813187" watchObservedRunningTime="2026-09-07 11:37:01.399902447 +0000 UTC m=+853.347253231"
Sep 07 11:37:12.389988 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:12.389952    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:14.401949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:14.401897    2560 patch_prober.go:28] interesting pod/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 11:37:14.402335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:14.401983    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 11:37:16.464397 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:16.464289    2560 generic.go:356] "Generic (PLEG): container finished" podID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerID="02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60" exitCode=0
Sep 07 11:37:16.464397 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:16.464376    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" event={"ID":"04c02e88-5ccd-461c-9778-0af1e485d4e7","Type":"ContainerDied","Data":"02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"}
Sep 07 11:37:16.464856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:16.464814    2560 scope.go:117] "RemoveContainer" containerID="02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"
Sep 07 11:37:16.466032 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:16.466004    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:16.466110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:16.466095    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:16.466147 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:16.466127    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:17.472959 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:17.472920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" event={"ID":"04c02e88-5ccd-461c-9778-0af1e485d4e7","Type":"ContainerStarted","Data":"f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5"}
Sep 07 11:37:18.202168 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:18.202109    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:37:18.202529 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:18.202497    2560 patch_prober.go:28] interesting pod/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 07 11:37:18.202648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:18.202581    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 07 11:37:19.202773 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:19.202706    2560 patch_prober.go:28] interesting pod/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 07 11:37:19.203283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:19.202811    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 07 11:37:20.202143 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:20.202096    2560 patch_prober.go:28] interesting pod/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 07 11:37:20.202351 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:20.202166    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 07 11:37:21.201942 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:21.201901    2560 patch_prober.go:28] interesting pod/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 07 11:37:21.202541 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:21.202500    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 07 11:37:22.206768 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:22.203180    2560 patch_prober.go:28] interesting pod/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 07 11:37:22.206768 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:22.203248    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 07 11:37:23.205156 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:23.205124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:37:23.205451 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:23.205428    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:37:23.206161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:23.206145    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:37:30.819438 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:30.819402    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"]
Sep 07 11:37:30.819923 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:30.819671    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6" podUID="3f98e78a-78d5-4c9a-850c-51ec15df4fbe" containerName="payload-processing" containerID="cri-o://11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4" gracePeriod=30
Sep 07 11:37:31.068394 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.068368    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:31.150275 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.150181    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"]
Sep 07 11:37:31.150553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.150499    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy" containerID="cri-o://f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5" gracePeriod=30
Sep 07 11:37:31.200562 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.200536    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtsmv\" (UniqueName: \"kubernetes.io/projected/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-kube-api-access-vtsmv\") pod \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") "
Sep 07 11:37:31.200726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.200632    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-plugins-config-volume\") pod \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\" (UID: \"3f98e78a-78d5-4c9a-850c-51ec15df4fbe\") "
Sep 07 11:37:31.201058 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.201025    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f98e78a-78d5-4c9a-850c-51ec15df4fbe" (UID: "3f98e78a-78d5-4c9a-850c-51ec15df4fbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:37:31.202652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.202621    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-kube-api-access-vtsmv" (OuterVolumeSpecName: "kube-api-access-vtsmv") pod "3f98e78a-78d5-4c9a-850c-51ec15df4fbe" (UID: "3f98e78a-78d5-4c9a-850c-51ec15df4fbe"). InnerVolumeSpecName "kube-api-access-vtsmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:37:31.301759 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.301711    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtsmv\" (UniqueName: \"kubernetes.io/projected/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-kube-api-access-vtsmv\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:31.301759 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.301756    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f98e78a-78d5-4c9a-850c-51ec15df4fbe-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:31.554591 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.554555    2560 generic.go:356] "Generic (PLEG): container finished" podID="3f98e78a-78d5-4c9a-850c-51ec15df4fbe" containerID="11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4" exitCode=0
Sep 07 11:37:31.554805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.554600    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"
Sep 07 11:37:31.554805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.554635    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6" event={"ID":"3f98e78a-78d5-4c9a-850c-51ec15df4fbe","Type":"ContainerDied","Data":"11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4"}
Sep 07 11:37:31.554805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.554685    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6" event={"ID":"3f98e78a-78d5-4c9a-850c-51ec15df4fbe","Type":"ContainerDied","Data":"779b02962d02a7d9c0e09c11bf6323360685db6ebc307212d3a29b4175bd6b28"}
Sep 07 11:37:31.554805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.554708    2560 scope.go:117] "RemoveContainer" containerID="11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4"
Sep 07 11:37:31.564318 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.564300    2560 scope.go:117] "RemoveContainer" containerID="11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4"
Sep 07 11:37:31.564663 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:37:31.564640    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4\": container with ID starting with 11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4 not found: ID does not exist" containerID="11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4"
Sep 07 11:37:31.564712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.564672    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4"} err="failed to get container status \"11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4\": rpc error: code = NotFound desc = could not find container \"11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4\": container with ID starting with 11c08bb659cf2b0a85e009bc8b75929627f09eae6ffd4f54504d251874a0c0a4 not found: ID does not exist"
Sep 07 11:37:31.576514 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.576489    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"]
Sep 07 11:37:31.580137 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.580109    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-25494d09-6c554d6774-vxkp6"]
Sep 07 11:37:31.922187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922107    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"]
Sep 07 11:37:31.922611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922595    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy"
Sep 07 11:37:31.922663 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922613    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy"
Sep 07 11:37:31.922663 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922625    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f98e78a-78d5-4c9a-850c-51ec15df4fbe" containerName="payload-processing"
Sep 07 11:37:31.922663 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922631    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f98e78a-78d5-4c9a-850c-51ec15df4fbe" containerName="payload-processing"
Sep 07 11:37:31.922663 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922642    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy"
Sep 07 11:37:31.922663 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922651    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy"
Sep 07 11:37:31.922862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922723    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f98e78a-78d5-4c9a-850c-51ec15df4fbe" containerName="payload-processing"
Sep 07 11:37:31.922862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922734    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy"
Sep 07 11:37:31.922862 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.922758    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="da065508-f26b-4431-9753-7a05d9ab1b47" containerName="istio-proxy"
Sep 07 11:37:31.926086 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.926064    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:31.928217 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.928195    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5e86f370-openshift-default-dockercfg-65zvj\""
Sep 07 11:37:31.939518 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:31.939498    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"]
Sep 07 11:37:32.007995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.007962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istiod-ca-cert\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.007995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.007998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-socket\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008069    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-token\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008115    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-certs\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-envoy\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-credential-socket\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-podinfo\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-data\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.008434 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.008385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rds6\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-kube-api-access-6rds6\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.109603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rds6\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-kube-api-access-6rds6\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.109792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istiod-ca-cert\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.109792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-socket\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.109792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-token\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.109792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-certs\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-envoy\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.109962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-credential-socket\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-podinfo\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110192 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-data\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110192 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110174    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-socket\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110304 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110205    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-certs\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110403 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-data\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110554 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istiod-ca-cert\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.110623 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.110561    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-credential-socket\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.112551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.112528    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-envoy\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.112780 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.112730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-podinfo\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.121164 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.121113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rds6\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-kube-api-access-6rds6\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.121283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.121166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-token\") pod \"e2e-ait-5e86f370-openshift-default-567877f884-q4xd8\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") " pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.235886 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.235857    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:32.497972 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.497897    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f98e78a-78d5-4c9a-850c-51ec15df4fbe" path="/var/lib/kubelet/pods/3f98e78a-78d5-4c9a-850c-51ec15df4fbe/volumes"
Sep 07 11:37:32.510415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.510393    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"]
Sep 07 11:37:32.512372 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:37:32.512343    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca86728_6b9d_44f3_aaef_5ea3f2b6c251.slice/crio-915166c838bf06832b68061cdf1f0a2726938865157bfe246939a98b5f9af689 WatchSource:0}: Error finding container 915166c838bf06832b68061cdf1f0a2726938865157bfe246939a98b5f9af689: Status 404 returned error can't find the container with id 915166c838bf06832b68061cdf1f0a2726938865157bfe246939a98b5f9af689
Sep 07 11:37:32.515006 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.514974    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:32.515081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.515039    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:32.515123 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.515080    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:32.562636 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:32.562606    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" event={"ID":"eca86728-6b9d-44f3-aaef-5ea3f2b6c251","Type":"ContainerStarted","Data":"915166c838bf06832b68061cdf1f0a2726938865157bfe246939a98b5f9af689"}
Sep 07 11:37:33.570068 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:33.570028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" event={"ID":"eca86728-6b9d-44f3-aaef-5ea3f2b6c251","Type":"ContainerStarted","Data":"675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"}
Sep 07 11:37:33.593808 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:33.593725    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podStartSLOduration=2.5937077779999997 podStartE2EDuration="2.593707778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:37:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:37:33.592594463 +0000 UTC m=+885.539945249" watchObservedRunningTime="2026-09-07 11:37:33.593707778 +0000 UTC m=+885.541058561"
Sep 07 11:37:34.236412 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:34.236376    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:34.241503 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:34.241464    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:34.575804 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:34.575696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:34.576890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:34.576868    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:36.398149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.398123    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:37:36.449350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449267    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmdp6\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-kube-api-access-mmdp6\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449329    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c02e88-5ccd-461c-9778-0af1e485d4e7-istiod-ca-cert\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449351    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-certs\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449371    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-envoy\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449388    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-token\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449402    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-socket\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449441    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-credential-socket\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449464    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-podinfo\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449513    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-data\") pod \"04c02e88-5ccd-461c-9778-0af1e485d4e7\" (UID: \"04c02e88-5ccd-461c-9778-0af1e485d4e7\") "
Sep 07 11:37:36.449957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449681    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:36.449957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449800    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:36.449957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.449863    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04c02e88-5ccd-461c-9778-0af1e485d4e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:37:36.450117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450043    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:36.450117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450105    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:36.450491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450467    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.450591 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450495    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.450591 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450510    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c02e88-5ccd-461c-9778-0af1e485d4e7-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.450591 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450525    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.450591 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.450551    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.451846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.451817    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:37:36.451949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.451892    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:37:36.452053 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.452033    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:37:36.452114 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.452068    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-kube-api-access-mmdp6" (OuterVolumeSpecName: "kube-api-access-mmdp6") pod "04c02e88-5ccd-461c-9778-0af1e485d4e7" (UID: "04c02e88-5ccd-461c-9778-0af1e485d4e7"). InnerVolumeSpecName "kube-api-access-mmdp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:37:36.551535 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.551504    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.551535 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.551527    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.551723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.551551    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c02e88-5ccd-461c-9778-0af1e485d4e7-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.551723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.551573    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmdp6\" (UniqueName: \"kubernetes.io/projected/04c02e88-5ccd-461c-9778-0af1e485d4e7-kube-api-access-mmdp6\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:37:36.587832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.587803    2560 generic.go:356] "Generic (PLEG): container finished" podID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerID="f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5" exitCode=0
Sep 07 11:37:36.587970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.587854    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"
Sep 07 11:37:36.587970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.587860    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" event={"ID":"04c02e88-5ccd-461c-9778-0af1e485d4e7","Type":"ContainerDied","Data":"f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5"}
Sep 07 11:37:36.587970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.587914    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr" event={"ID":"04c02e88-5ccd-461c-9778-0af1e485d4e7","Type":"ContainerDied","Data":"70b03978cfb41bc75d6e2d52f89e21f8aa2ad7638066748dce56b423f01c0111"}
Sep 07 11:37:36.587970 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.587938    2560 scope.go:117] "RemoveContainer" containerID="f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5"
Sep 07 11:37:36.596728 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.596711    2560 scope.go:117] "RemoveContainer" containerID="02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"
Sep 07 11:37:36.605279 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.605257    2560 scope.go:117] "RemoveContainer" containerID="f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5"
Sep 07 11:37:36.605555 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:37:36.605537    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5\": container with ID starting with f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5 not found: ID does not exist" containerID="f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5"
Sep 07 11:37:36.605615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.605563    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5"} err="failed to get container status \"f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5\": rpc error: code = NotFound desc = could not find container \"f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5\": container with ID starting with f59c791b6554e22e68532cd198417a11c9a1fc2c3d20c64ef379ef53232907f5 not found: ID does not exist"
Sep 07 11:37:36.605615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.605580    2560 scope.go:117] "RemoveContainer" containerID="02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"
Sep 07 11:37:36.605722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.605644    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"]
Sep 07 11:37:36.605889 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:37:36.605873    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60\": container with ID starting with 02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60 not found: ID does not exist" containerID="02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"
Sep 07 11:37:36.605935 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.605896    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60"} err="failed to get container status \"02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60\": rpc error: code = NotFound desc = could not find container \"02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60\": container with ID starting with 02fb16660556f5e00448a2e5bc2287ea9633c59809ae2dfbac61676e563c1d60 not found: ID does not exist"
Sep 07 11:37:36.611807 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.611785    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-25494d09-gw-openshift-default-7b46b89cbd-58mrr"]
Sep 07 11:37:36.744973 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.744938    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"]
Sep 07 11:37:36.745357 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.745344    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy"
Sep 07 11:37:36.745402 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.745358    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy"
Sep 07 11:37:36.745444 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.745439    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy"
Sep 07 11:37:36.745489 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.745448    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy"
Sep 07 11:37:36.749037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.749021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:36.751231 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.751196    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5e86f370\""
Sep 07 11:37:36.751364 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.751306    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5e86f370-dockercfg-v55j9\""
Sep 07 11:37:36.761270 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.761246    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"]
Sep 07 11:37:36.854128 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.854086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-plugins-config-volume\") pod \"payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") " pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:36.854289 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.854137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4dzc\" (UniqueName: \"kubernetes.io/projected/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-kube-api-access-x4dzc\") pod \"payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") " pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:36.955640 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.955596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-plugins-config-volume\") pod \"payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") " pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:36.955855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.955656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4dzc\" (UniqueName: \"kubernetes.io/projected/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-kube-api-access-x4dzc\") pod \"payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") " pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:36.956423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.956398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-plugins-config-volume\") pod \"payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") " pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:36.967495 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:36.967472    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4dzc\" (UniqueName: \"kubernetes.io/projected/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-kube-api-access-x4dzc\") pod \"payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") " pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:37.060640 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:37.060554    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:37.341008 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:37.340980    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"]
Sep 07 11:37:37.343062 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:37:37.343017    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64a5c53_6bb5_4a41_869f_a0882ee0ded7.slice/crio-061208ed1902737e85b884ed56a73a39dc6e9b5c650dbd3d11cef22750b8c165 WatchSource:0}: Error finding container 061208ed1902737e85b884ed56a73a39dc6e9b5c650dbd3d11cef22750b8c165: Status 404 returned error can't find the container with id 061208ed1902737e85b884ed56a73a39dc6e9b5c650dbd3d11cef22750b8c165
Sep 07 11:37:37.596282 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:37.596183    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9" event={"ID":"d64a5c53-6bb5-4a41-869f-a0882ee0ded7","Type":"ContainerStarted","Data":"08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b"}
Sep 07 11:37:37.596282 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:37.596247    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:37.596282 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:37.596270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9" event={"ID":"d64a5c53-6bb5-4a41-869f-a0882ee0ded7","Type":"ContainerStarted","Data":"061208ed1902737e85b884ed56a73a39dc6e9b5c650dbd3d11cef22750b8c165"}
Sep 07 11:37:37.617399 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:37.617345    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9" podStartSLOduration=1.617330827 podStartE2EDuration="1.617330827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:37:37.615141862 +0000 UTC m=+889.562492652" watchObservedRunningTime="2026-09-07 11:37:37.617330827 +0000 UTC m=+889.564681612"
Sep 07 11:37:38.501419 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:38.501376    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" path="/var/lib/kubelet/pods/04c02e88-5ccd-461c-9778-0af1e485d4e7/volumes"
Sep 07 11:37:48.520645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:48.520556    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:37:48.521369 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:48.521349    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:37:48.605983 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:48.605943    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:37:57.710800 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:57.710766    2560 generic.go:356] "Generic (PLEG): container finished" podID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerID="675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13" exitCode=0
Sep 07 11:37:57.711225 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:57.710824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" event={"ID":"eca86728-6b9d-44f3-aaef-5ea3f2b6c251","Type":"ContainerDied","Data":"675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"}
Sep 07 11:37:57.711317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:57.711299    2560 scope.go:117] "RemoveContainer" containerID="675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"
Sep 07 11:37:57.712457 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:57.712423    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:57.712514 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:57.712498    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:57.712553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:57.712535    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:37:58.720546 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:58.720506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" event={"ID":"eca86728-6b9d-44f3-aaef-5ea3f2b6c251","Type":"ContainerStarted","Data":"a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8"}
Sep 07 11:37:59.236679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:59.236642    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:37:59.236966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:59.236942    2560 patch_prober.go:28] interesting pod/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 07 11:37:59.237019 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:37:59.236996    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 07 11:38:00.237120 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:00.237073    2560 patch_prober.go:28] interesting pod/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 07 11:38:00.237611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:00.237147    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 07 11:38:01.236787 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:01.236726    2560 patch_prober.go:28] interesting pod/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 07 11:38:01.236985 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:01.236809    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 07 11:38:02.236826 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:02.236792    2560 patch_prober.go:28] interesting pod/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 07 11:38:02.237206 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:02.236843    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 07 11:38:03.236420 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:03.236380    2560 patch_prober.go:28] interesting pod/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 07 11:38:03.236617 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:03.236437    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 07 11:38:04.240533 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:04.240497    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:38:04.240960 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:04.240784    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:38:04.241561 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:04.241544    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:38:06.762380 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:06.762332    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"]
Sep 07 11:38:06.762877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:06.762606    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9" podUID="d64a5c53-6bb5-4a41-869f-a0882ee0ded7" containerName="payload-processing" containerID="cri-o://08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b" gracePeriod=30
Sep 07 11:38:07.030399 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.030159    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:38:07.078875 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.078847    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4dzc\" (UniqueName: \"kubernetes.io/projected/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-kube-api-access-x4dzc\") pod \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") "
Sep 07 11:38:07.079075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.078890    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-plugins-config-volume\") pod \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\" (UID: \"d64a5c53-6bb5-4a41-869f-a0882ee0ded7\") "
Sep 07 11:38:07.079285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.079259    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d64a5c53-6bb5-4a41-869f-a0882ee0ded7" (UID: "d64a5c53-6bb5-4a41-869f-a0882ee0ded7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:38:07.080985 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.080960    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-kube-api-access-x4dzc" (OuterVolumeSpecName: "kube-api-access-x4dzc") pod "d64a5c53-6bb5-4a41-869f-a0882ee0ded7" (UID: "d64a5c53-6bb5-4a41-869f-a0882ee0ded7"). InnerVolumeSpecName "kube-api-access-x4dzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:07.179593 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.179551    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4dzc\" (UniqueName: \"kubernetes.io/projected/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-kube-api-access-x4dzc\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:07.179593 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.179587    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d64a5c53-6bb5-4a41-869f-a0882ee0ded7-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:07.772415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.772380    2560 generic.go:356] "Generic (PLEG): container finished" podID="d64a5c53-6bb5-4a41-869f-a0882ee0ded7" containerID="08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b" exitCode=0
Sep 07 11:38:07.772870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.772424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9" event={"ID":"d64a5c53-6bb5-4a41-869f-a0882ee0ded7","Type":"ContainerDied","Data":"08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b"}
Sep 07 11:38:07.772870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.772460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9" event={"ID":"d64a5c53-6bb5-4a41-869f-a0882ee0ded7","Type":"ContainerDied","Data":"061208ed1902737e85b884ed56a73a39dc6e9b5c650dbd3d11cef22750b8c165"}
Sep 07 11:38:07.772870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.772480    2560 scope.go:117] "RemoveContainer" containerID="08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b"
Sep 07 11:38:07.772870 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.772425    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"
Sep 07 11:38:07.781729 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.781710    2560 scope.go:117] "RemoveContainer" containerID="08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b"
Sep 07 11:38:07.782053 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:07.782032    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b\": container with ID starting with 08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b not found: ID does not exist" containerID="08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b"
Sep 07 11:38:07.782103 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.782062    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b"} err="failed to get container status \"08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b\": rpc error: code = NotFound desc = could not find container \"08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b\": container with ID starting with 08d25810b3ad47e75c1360addb3c595daf74a4e5f836c1275cf778fb3c8f662b not found: ID does not exist"
Sep 07 11:38:07.795037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.795005    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"]
Sep 07 11:38:07.798076 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:07.798049    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e86f370-78f98684f9-k6jc9"]
Sep 07 11:38:08.260991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.260962    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"]
Sep 07 11:38:08.261194 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.261167    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy" containerID="cri-o://a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8" gracePeriod=30
Sep 07 11:38:08.498616 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.498585    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d64a5c53-6bb5-4a41-869f-a0882ee0ded7" path="/var/lib/kubelet/pods/d64a5c53-6bb5-4a41-869f-a0882ee0ded7/volumes"
Sep 07 11:38:08.994254 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.994219    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"]
Sep 07 11:38:08.994694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.994680    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy"
Sep 07 11:38:08.994795 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.994696    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="04c02e88-5ccd-461c-9778-0af1e485d4e7" containerName="istio-proxy"
Sep 07 11:38:08.994795 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.994726    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d64a5c53-6bb5-4a41-869f-a0882ee0ded7" containerName="payload-processing"
Sep 07 11:38:08.994795 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.994735    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d64a5c53-6bb5-4a41-869f-a0882ee0ded7" containerName="payload-processing"
Sep 07 11:38:08.994974 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.994847    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d64a5c53-6bb5-4a41-869f-a0882ee0ded7" containerName="payload-processing"
Sep 07 11:38:08.998426 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:08.998406    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.000407 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.000385    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3f8729d3-openshift-default-dockercfg-dtgn4\""
Sep 07 11:38:09.009661 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.009017    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"]
Sep 07 11:38:09.097644 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097610    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-podinfo\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.097856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-token\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.097856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097688    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-credential-socket\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.097856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-certs\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.097856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-socket\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.098007 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-envoy\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.098007 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8mmx\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-kube-api-access-s8mmx\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.098007 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.097988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-data\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.098105 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.098018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c2f135-d5c7-4ff2-ae57-8653af329a25-istiod-ca-cert\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.199614 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-data\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.199822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c2f135-d5c7-4ff2-ae57-8653af329a25-istiod-ca-cert\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.199822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-podinfo\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.199822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-token\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.199822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-credential-socket\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.199822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-certs\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-socket\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-envoy\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.199920    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8mmx\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-kube-api-access-s8mmx\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200175 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.200134    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-credential-socket\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200238 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.200150    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-data\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200238 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.200222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-certs\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.200267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-socket\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.200717 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.200690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c2f135-d5c7-4ff2-ae57-8653af329a25-istiod-ca-cert\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.202254 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.202231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-envoy\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.202425 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.202411    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-podinfo\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.207451 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.207430    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-token\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.207572 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.207555    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8mmx\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-kube-api-access-s8mmx\") pod \"e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") " pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.335823 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.335699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:09.615475 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.615446    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"]
Sep 07 11:38:09.616995 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:38:09.616970    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c2f135_d5c7_4ff2_ae57_8653af329a25.slice/crio-ad6c75edbe92db8731f238dac527e4f591d42e245c94f1c5533cc5075fce8d14 WatchSource:0}: Error finding container ad6c75edbe92db8731f238dac527e4f591d42e245c94f1c5533cc5075fce8d14: Status 404 returned error can't find the container with id ad6c75edbe92db8731f238dac527e4f591d42e245c94f1c5533cc5075fce8d14
Sep 07 11:38:09.619178 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.619144    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:09.619436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.619211    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:09.619436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.619242    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:09.786693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.786647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" event={"ID":"31c2f135-d5c7-4ff2-ae57-8653af329a25","Type":"ContainerStarted","Data":"12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"}
Sep 07 11:38:09.786903 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.786779    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" event={"ID":"31c2f135-d5c7-4ff2-ae57-8653af329a25","Type":"ContainerStarted","Data":"ad6c75edbe92db8731f238dac527e4f591d42e245c94f1c5533cc5075fce8d14"}
Sep 07 11:38:09.808012 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:09.807955    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podStartSLOduration=1.807936442 podStartE2EDuration="1.807936442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:38:09.807599041 +0000 UTC m=+921.754949826" watchObservedRunningTime="2026-09-07 11:38:09.807936442 +0000 UTC m=+921.755287233"
Sep 07 11:38:10.336880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:10.336842    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:10.342674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:10.342649    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:10.791814 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:10.791783    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:10.792573 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:10.792556    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:13.213411 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.213380    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"]
Sep 07 11:38:13.216415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.216393    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.218834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.218809    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-3f8729d3-dockercfg-n7w67\""
Sep 07 11:38:13.218919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.218820    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3f8729d3\""
Sep 07 11:38:13.226918 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.226895    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"]
Sep 07 11:38:13.240910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.240883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6q5v\" (UniqueName: \"kubernetes.io/projected/0b8260aa-6c78-4405-9a3e-b0db19a4216f-kube-api-access-q6q5v\") pod \"payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") " pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.240988 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.240968    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b8260aa-6c78-4405-9a3e-b0db19a4216f-plugins-config-volume\") pod \"payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") " pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.342506 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.342471    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b8260aa-6c78-4405-9a3e-b0db19a4216f-plugins-config-volume\") pod \"payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") " pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.342703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.342589    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6q5v\" (UniqueName: \"kubernetes.io/projected/0b8260aa-6c78-4405-9a3e-b0db19a4216f-kube-api-access-q6q5v\") pod \"payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") " pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.343446 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.343416    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b8260aa-6c78-4405-9a3e-b0db19a4216f-plugins-config-volume\") pod \"payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") " pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.351871 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.351840    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6q5v\" (UniqueName: \"kubernetes.io/projected/0b8260aa-6c78-4405-9a3e-b0db19a4216f-kube-api-access-q6q5v\") pod \"payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") " pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.528840 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.528810    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:38:13.530171 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.530148    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:13.645526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645498    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-certs\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645702 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645550    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istiod-ca-cert\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645702 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645587    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-token\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645702 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645624    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-podinfo\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645702 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645673    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-envoy\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645731    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-data\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645801    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rds6\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-kube-api-access-6rds6\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645832    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:13.645940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645839    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-socket\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.645940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.645913    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-credential-socket\") pod \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\" (UID: \"eca86728-6b9d-44f3-aaef-5ea3f2b6c251\") "
Sep 07 11:38:13.646188 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646018    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:13.646188 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646154    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:38:13.646388 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646363    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:13.646463 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646382    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.646463 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646406    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.646463 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646420    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.646463 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.646381    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-data" (OuterVolumeSpecName: "istio-data") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:13.648075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.648036    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-token" (OuterVolumeSpecName: "istio-token") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:13.648189 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.648112    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-kube-api-access-6rds6" (OuterVolumeSpecName: "kube-api-access-6rds6") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "kube-api-access-6rds6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:13.648311 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.648297    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:38:13.648756 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.648723    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eca86728-6b9d-44f3-aaef-5ea3f2b6c251" (UID: "eca86728-6b9d-44f3-aaef-5ea3f2b6c251"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:13.754476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.748380    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.754476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.748418    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.754476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.748440    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.754476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.748452    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.754476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.748466    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rds6\" (UniqueName: \"kubernetes.io/projected/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-kube-api-access-6rds6\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.754476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.748484    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca86728-6b9d-44f3-aaef-5ea3f2b6c251-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:13.811864 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.811834    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"]
Sep 07 11:38:13.812157 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.812133    2560 generic.go:356] "Generic (PLEG): container finished" podID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerID="a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8" exitCode=0
Sep 07 11:38:13.812201 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.812185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" event={"ID":"eca86728-6b9d-44f3-aaef-5ea3f2b6c251","Type":"ContainerDied","Data":"a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8"}
Sep 07 11:38:13.812244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.812222    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8" event={"ID":"eca86728-6b9d-44f3-aaef-5ea3f2b6c251","Type":"ContainerDied","Data":"915166c838bf06832b68061cdf1f0a2726938865157bfe246939a98b5f9af689"}
Sep 07 11:38:13.812277 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.812250    2560 scope.go:117] "RemoveContainer" containerID="a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8"
Sep 07 11:38:13.812432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.812418    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"
Sep 07 11:38:13.815580 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:38:13.815552    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8260aa_6c78_4405_9a3e_b0db19a4216f.slice/crio-86844daaa96dcbc299828a65035ec035406b6f362fd36b25f6265038f07473bd WatchSource:0}: Error finding container 86844daaa96dcbc299828a65035ec035406b6f362fd36b25f6265038f07473bd: Status 404 returned error can't find the container with id 86844daaa96dcbc299828a65035ec035406b6f362fd36b25f6265038f07473bd
Sep 07 11:38:13.825808 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.825786    2560 scope.go:117] "RemoveContainer" containerID="675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"
Sep 07 11:38:13.839187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.839158    2560 scope.go:117] "RemoveContainer" containerID="a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8"
Sep 07 11:38:13.839615 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:13.839551    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8\": container with ID starting with a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8 not found: ID does not exist" containerID="a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8"
Sep 07 11:38:13.839991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.839950    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8"} err="failed to get container status \"a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8\": rpc error: code = NotFound desc = could not find container \"a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8\": container with ID starting with a65406d4125ab0d045c5725b4503377d62ce10a8d9070dd9f87d9da61c5a31a8 not found: ID does not exist"
Sep 07 11:38:13.840096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.839994    2560 scope.go:117] "RemoveContainer" containerID="675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"
Sep 07 11:38:13.840379 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:13.840349    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13\": container with ID starting with 675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13 not found: ID does not exist" containerID="675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"
Sep 07 11:38:13.840464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.840388    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13"} err="failed to get container status \"675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13\": rpc error: code = NotFound desc = could not find container \"675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13\": container with ID starting with 675f5dd90c94657ca40387deea02265cb26ea6f733c6c708ab9af33ed538ff13 not found: ID does not exist"
Sep 07 11:38:13.840533 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.840473    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"]
Sep 07 11:38:13.846449 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:13.846422    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5e86f370-openshift-default-567877f884-q4xd8"]
Sep 07 11:38:14.497731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:14.497699    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" path="/var/lib/kubelet/pods/eca86728-6b9d-44f3-aaef-5ea3f2b6c251/volumes"
Sep 07 11:38:14.820641 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:14.820610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc" event={"ID":"0b8260aa-6c78-4405-9a3e-b0db19a4216f","Type":"ContainerStarted","Data":"d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13"}
Sep 07 11:38:14.820839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:14.820657    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc" event={"ID":"0b8260aa-6c78-4405-9a3e-b0db19a4216f","Type":"ContainerStarted","Data":"86844daaa96dcbc299828a65035ec035406b6f362fd36b25f6265038f07473bd"}
Sep 07 11:38:14.820839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:14.820750    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:14.838388 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:14.838346    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc" podStartSLOduration=1.838330988 podStartE2EDuration="1.838330988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:38:14.836640087 +0000 UTC m=+926.783990871" watchObservedRunningTime="2026-09-07 11:38:14.838330988 +0000 UTC m=+926.785681832"
Sep 07 11:38:25.828471 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:25.828433    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:32.002398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.002355    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"]
Sep 07 11:38:32.923926 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.923896    2560 generic.go:356] "Generic (PLEG): container finished" podID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerID="12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835" exitCode=0
Sep 07 11:38:32.924085 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.923937    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" event={"ID":"31c2f135-d5c7-4ff2-ae57-8653af329a25","Type":"ContainerDied","Data":"12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"}
Sep 07 11:38:32.924357 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.924342    2560 scope.go:117] "RemoveContainer" containerID="12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"
Sep 07 11:38:32.925520 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.925485    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:32.925771 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.925571    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:32.925771 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:32.925621    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:33.932787 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:33.932733    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" event={"ID":"31c2f135-d5c7-4ff2-ae57-8653af329a25","Type":"ContainerStarted","Data":"aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d"}
Sep 07 11:38:34.336593 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:34.336559    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:34.336922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:34.336899    2560 patch_prober.go:28] interesting pod/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 07 11:38:34.336994 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:34.336950    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 07 11:38:35.337111 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:35.337072    2560 patch_prober.go:28] interesting pod/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 07 11:38:35.337486 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:35.337140    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 07 11:38:36.040799 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.040756    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"]
Sep 07 11:38:36.041217 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.041203    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy"
Sep 07 11:38:36.041263 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.041220    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy"
Sep 07 11:38:36.041263 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.041232    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy"
Sep 07 11:38:36.041263 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.041238    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy"
Sep 07 11:38:36.041365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.041311    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy"
Sep 07 11:38:36.041365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.041319    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca86728-6b9d-44f3-aaef-5ea3f2b6c251" containerName="istio-proxy"
Sep 07 11:38:36.043893 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.043876    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:36.046080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.046052    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:38:36.052079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.052060    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"]
Sep 07 11:38:36.186677 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.186639    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2a18336-4d80-470a-aab8-2efa15611b00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt\" (UID: \"a2a18336-4d80-470a-aab8-2efa15611b00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:36.287405 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.287369    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2a18336-4d80-470a-aab8-2efa15611b00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt\" (UID: \"a2a18336-4d80-470a-aab8-2efa15611b00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:36.287984 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.287966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2a18336-4d80-470a-aab8-2efa15611b00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt\" (UID: \"a2a18336-4d80-470a-aab8-2efa15611b00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:36.336415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.336330    2560 patch_prober.go:28] interesting pod/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 07 11:38:36.336567 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.336406    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 07 11:38:36.355729 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.355696    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:36.655436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.655409    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"]
Sep 07 11:38:36.657719 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:38:36.657685    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a18336_4d80_470a_aab8_2efa15611b00.slice/crio-b7f9667566e6c69687d5d75c3bb7413fe2be30238a6ee34f737464ed94362ed1 WatchSource:0}: Error finding container b7f9667566e6c69687d5d75c3bb7413fe2be30238a6ee34f737464ed94362ed1: Status 404 returned error can't find the container with id b7f9667566e6c69687d5d75c3bb7413fe2be30238a6ee34f737464ed94362ed1
Sep 07 11:38:36.659720 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.659704    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:38:36.949020 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:36.948929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt" event={"ID":"a2a18336-4d80-470a-aab8-2efa15611b00","Type":"ContainerStarted","Data":"b7f9667566e6c69687d5d75c3bb7413fe2be30238a6ee34f737464ed94362ed1"}
Sep 07 11:38:37.337048 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:37.337007    2560 patch_prober.go:28] interesting pod/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 07 11:38:37.337223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:37.337073    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 07 11:38:38.337096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:38.337055    2560 patch_prober.go:28] interesting pod/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 07 11:38:38.337476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:38.337120    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 07 11:38:39.467918 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:39.467887    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:39.468800 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:39.468772    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:39.469904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:39.469880    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:40.974107 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:40.974009    2560 generic.go:356] "Generic (PLEG): container finished" podID="a2a18336-4d80-470a-aab8-2efa15611b00" containerID="66cd139cc494df04f5241a4870a20b3d83637868e1f879152bd49b4873a63880" exitCode=0
Sep 07 11:38:40.974482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:40.974099    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt" event={"ID":"a2a18336-4d80-470a-aab8-2efa15611b00","Type":"ContainerDied","Data":"66cd139cc494df04f5241a4870a20b3d83637868e1f879152bd49b4873a63880"}
Sep 07 11:38:42.115679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.115655    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:42.157432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.157398    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2a18336-4d80-470a-aab8-2efa15611b00-maas-api-service-ca\") pod \"a2a18336-4d80-470a-aab8-2efa15611b00\" (UID: \"a2a18336-4d80-470a-aab8-2efa15611b00\") "
Sep 07 11:38:42.157709 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.157688    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2a18336-4d80-470a-aab8-2efa15611b00-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a2a18336-4d80-470a-aab8-2efa15611b00" (UID: "a2a18336-4d80-470a-aab8-2efa15611b00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:38:42.157828 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.157815    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2a18336-4d80-470a-aab8-2efa15611b00-maas-api-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:42.986370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.986342    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"
Sep 07 11:38:42.986370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.986352    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt" event={"ID":"a2a18336-4d80-470a-aab8-2efa15611b00","Type":"ContainerDied","Data":"b7f9667566e6c69687d5d75c3bb7413fe2be30238a6ee34f737464ed94362ed1"}
Sep 07 11:38:42.986588 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:42.986396    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7f9667566e6c69687d5d75c3bb7413fe2be30238a6ee34f737464ed94362ed1"
Sep 07 11:38:47.023643 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.023601    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m" podUID="bd8b25b7-a7b8-490f-a62e-c0559ea533a8" containerName="main" containerID="cri-o://6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920" gracePeriod=45
Sep 07 11:38:47.286966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.286898    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:38:47.304629 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304603    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tmp-dir\") pod \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") "
Sep 07 11:38:47.304782 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304718    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tls-certs\") pod \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") "
Sep 07 11:38:47.304782 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304764    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-home\") pod \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") "
Sep 07 11:38:47.304877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304810    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-dshm\") pod \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") "
Sep 07 11:38:47.304877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304836    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-model-cache\") pod \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") "
Sep 07 11:38:47.304877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd8b25b7-a7b8-490f-a62e-c0559ea533a8" (UID: "bd8b25b7-a7b8-490f-a62e-c0559ea533a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:47.304877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.304865    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fq7t\" (UniqueName: \"kubernetes.io/projected/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-kube-api-access-7fq7t\") pod \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\" (UID: \"bd8b25b7-a7b8-490f-a62e-c0559ea533a8\") "
Sep 07 11:38:47.305054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.305032    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd8b25b7-a7b8-490f-a62e-c0559ea533a8" (UID: "bd8b25b7-a7b8-490f-a62e-c0559ea533a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:47.305111 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.305085    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-home" (OuterVolumeSpecName: "home") pod "bd8b25b7-a7b8-490f-a62e-c0559ea533a8" (UID: "bd8b25b7-a7b8-490f-a62e-c0559ea533a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:47.305228 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.305212    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-home\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:47.305290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.305229    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-model-cache\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:47.305290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.305258    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tmp-dir\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:47.307074 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.307013    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd8b25b7-a7b8-490f-a62e-c0559ea533a8" (UID: "bd8b25b7-a7b8-490f-a62e-c0559ea533a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:38:47.307215 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.307121    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-dshm" (OuterVolumeSpecName: "dshm") pod "bd8b25b7-a7b8-490f-a62e-c0559ea533a8" (UID: "bd8b25b7-a7b8-490f-a62e-c0559ea533a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:47.307300 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.307197    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-kube-api-access-7fq7t" (OuterVolumeSpecName: "kube-api-access-7fq7t") pod "bd8b25b7-a7b8-490f-a62e-c0559ea533a8" (UID: "bd8b25b7-a7b8-490f-a62e-c0559ea533a8"). InnerVolumeSpecName "kube-api-access-7fq7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:47.406373 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.406330    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-tls-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:47.406373 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.406365    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-dshm\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:47.406373 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:47.406379    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fq7t\" (UniqueName: \"kubernetes.io/projected/bd8b25b7-a7b8-490f-a62e-c0559ea533a8-kube-api-access-7fq7t\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:48.013497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.013463    2560 generic.go:356] "Generic (PLEG): container finished" podID="bd8b25b7-a7b8-490f-a62e-c0559ea533a8" containerID="6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920" exitCode=0
Sep 07 11:38:48.013686 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.013513    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"
Sep 07 11:38:48.013686 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.013536    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m" event={"ID":"bd8b25b7-a7b8-490f-a62e-c0559ea533a8","Type":"ContainerDied","Data":"6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920"}
Sep 07 11:38:48.013686 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.013585    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m" event={"ID":"bd8b25b7-a7b8-490f-a62e-c0559ea533a8","Type":"ContainerDied","Data":"c44f506946b0543d40deda3a205f8d8556f065235a077832f8acd6276d80ef42"}
Sep 07 11:38:48.013686 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.013613    2560 scope.go:117] "RemoveContainer" containerID="6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920"
Sep 07 11:38:48.022949 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.022926    2560 scope.go:117] "RemoveContainer" containerID="6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920"
Sep 07 11:38:48.023223 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:48.023202    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920\": container with ID starting with 6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920 not found: ID does not exist" containerID="6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920"
Sep 07 11:38:48.023277 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.023232    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920"} err="failed to get container status \"6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920\": rpc error: code = NotFound desc = could not find container \"6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920\": container with ID starting with 6a13b7d0a20856ab78bc87c9de684a9747ae4fbbc136ed28f1d704509d1d6920 not found: ID does not exist"
Sep 07 11:38:48.036110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.036060    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"]
Sep 07 11:38:48.038127 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.038107    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-6580730c-kserve-58f79b6cdc-wbf5m"]
Sep 07 11:38:48.497955 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:48.497912    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8b25b7-a7b8-490f-a62e-c0559ea533a8" path="/var/lib/kubelet/pods/bd8b25b7-a7b8-490f-a62e-c0559ea533a8/volumes"
Sep 07 11:38:52.049286 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049252    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"]
Sep 07 11:38:52.049766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049675    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2a18336-4d80-470a-aab8-2efa15611b00" containerName="revoke-keys"
Sep 07 11:38:52.049766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049688    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2a18336-4d80-470a-aab8-2efa15611b00" containerName="revoke-keys"
Sep 07 11:38:52.049766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049700    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8b25b7-a7b8-490f-a62e-c0559ea533a8" containerName="main"
Sep 07 11:38:52.049766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049706    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8b25b7-a7b8-490f-a62e-c0559ea533a8" containerName="main"
Sep 07 11:38:52.049978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049787    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2a18336-4d80-470a-aab8-2efa15611b00" containerName="revoke-keys"
Sep 07 11:38:52.049978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.049797    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8b25b7-a7b8-490f-a62e-c0559ea533a8" containerName="main"
Sep 07 11:38:52.053589 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.053568    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.055917 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.055894    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-bf198c-openshift-default-dockercfg-5tj4k\""
Sep 07 11:38:52.069364 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.069336    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"]
Sep 07 11:38:52.151829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.151788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrrf\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-kube-api-access-nzrrf\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.151851    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-certs\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.151928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-credential-socket\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.151963    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istiod-ca-cert\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.152018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-envoy\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.152043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-socket\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.152079    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-token\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.152166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-podinfo\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.152348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.152279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-data\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-envoy\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-socket\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-token\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253517    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-podinfo\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253574    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-data\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253648    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrrf\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-kube-api-access-nzrrf\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-certs\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-credential-socket\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istiod-ca-cert\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.253967 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.253954    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-socket\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.254183 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.254030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-data\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.254183 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.254136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-certs\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.254299 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.254219    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-credential-socket\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.254541 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.254517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istiod-ca-cert\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.255909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.255887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-envoy\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.256010 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.255934    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-podinfo\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.262031 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.262009    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrrf\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-kube-api-access-nzrrf\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.262132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.262061    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-token\") pod \"e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") " pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.364299 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.364203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:52.639924 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.639895    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"]
Sep 07 11:38:52.641702 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:38:52.641676    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520f0ab4_c0d9_4d1b_b7ae_d80eaac7be5a.slice/crio-9ad6856a1e4b0725cf28c2410a6cb6b21f2cd96867fdda0b2171c94ad0d6b007 WatchSource:0}: Error finding container 9ad6856a1e4b0725cf28c2410a6cb6b21f2cd96867fdda0b2171c94ad0d6b007: Status 404 returned error can't find the container with id 9ad6856a1e4b0725cf28c2410a6cb6b21f2cd96867fdda0b2171c94ad0d6b007
Sep 07 11:38:52.643974 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.643936    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:52.644060 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.644011    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:52.644060 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:52.644051    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:38:53.043484 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.043445    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" event={"ID":"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a","Type":"ContainerStarted","Data":"a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"}
Sep 07 11:38:53.043665 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.043502    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" event={"ID":"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a","Type":"ContainerStarted","Data":"9ad6856a1e4b0725cf28c2410a6cb6b21f2cd96867fdda0b2171c94ad0d6b007"}
Sep 07 11:38:53.065524 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.065467    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podStartSLOduration=1.065451256 podStartE2EDuration="1.065451256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:38:53.063770233 +0000 UTC m=+965.011121018" watchObservedRunningTime="2026-09-07 11:38:53.065451256 +0000 UTC m=+965.012802040"
Sep 07 11:38:53.344601 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.344513    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"]
Sep 07 11:38:53.344855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.344792    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc" podUID="0b8260aa-6c78-4405-9a3e-b0db19a4216f" containerName="payload-processing" containerID="cri-o://d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13" gracePeriod=30
Sep 07 11:38:53.368919 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.368879    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 07 11:38:53.369080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.368947    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 07 11:38:53.369327 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.369215    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:53.533586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.533544    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"]
Sep 07 11:38:53.535833 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.535802    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3f8729d3-swvzt"]
Sep 07 11:38:53.605580 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.605518    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:53.699762 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.697189    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"]
Sep 07 11:38:53.699762 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.697427    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" containerID="cri-o://aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d" gracePeriod=30
Sep 07 11:38:53.718938 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.716278    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b8260aa-6c78-4405-9a3e-b0db19a4216f-plugins-config-volume\") pod \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") "
Sep 07 11:38:53.728760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.721102    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b8260aa-6c78-4405-9a3e-b0db19a4216f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b8260aa-6c78-4405-9a3e-b0db19a4216f" (UID: "0b8260aa-6c78-4405-9a3e-b0db19a4216f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:38:53.728760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.721243    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6q5v\" (UniqueName: \"kubernetes.io/projected/0b8260aa-6c78-4405-9a3e-b0db19a4216f-kube-api-access-q6q5v\") pod \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\" (UID: \"0b8260aa-6c78-4405-9a3e-b0db19a4216f\") "
Sep 07 11:38:53.728760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.721593    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b8260aa-6c78-4405-9a3e-b0db19a4216f-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:53.744137 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.744104    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b8260aa-6c78-4405-9a3e-b0db19a4216f-kube-api-access-q6q5v" (OuterVolumeSpecName: "kube-api-access-q6q5v") pod "0b8260aa-6c78-4405-9a3e-b0db19a4216f" (UID: "0b8260aa-6c78-4405-9a3e-b0db19a4216f"). InnerVolumeSpecName "kube-api-access-q6q5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:53.822365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:53.822332    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6q5v\" (UniqueName: \"kubernetes.io/projected/0b8260aa-6c78-4405-9a3e-b0db19a4216f-kube-api-access-q6q5v\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:54.049857 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.049815    2560 generic.go:356] "Generic (PLEG): container finished" podID="0b8260aa-6c78-4405-9a3e-b0db19a4216f" containerID="d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13" exitCode=0
Sep 07 11:38:54.050045 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.049865    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"
Sep 07 11:38:54.050045 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.049891    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc" event={"ID":"0b8260aa-6c78-4405-9a3e-b0db19a4216f","Type":"ContainerDied","Data":"d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13"}
Sep 07 11:38:54.050045 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.049950    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc" event={"ID":"0b8260aa-6c78-4405-9a3e-b0db19a4216f","Type":"ContainerDied","Data":"86844daaa96dcbc299828a65035ec035406b6f362fd36b25f6265038f07473bd"}
Sep 07 11:38:54.050045 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.049972    2560 scope.go:117] "RemoveContainer" containerID="d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13"
Sep 07 11:38:54.058947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.058924    2560 scope.go:117] "RemoveContainer" containerID="d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13"
Sep 07 11:38:54.059225 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:54.059205    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13\": container with ID starting with d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13 not found: ID does not exist" containerID="d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13"
Sep 07 11:38:54.059277 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.059232    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13"} err="failed to get container status \"d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13\": rpc error: code = NotFound desc = could not find container \"d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13\": container with ID starting with d68c3052d2d65143671797a78a4c0a3d389dd90ddab43a79e6fa69c68470fe13 not found: ID does not exist"
Sep 07 11:38:54.074083 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.074046    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"]
Sep 07 11:38:54.088418 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.088376    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3f8729d3-56df688cfc-6dcfc"]
Sep 07 11:38:54.370320 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.370241    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:54.471297 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.471257    2560 patch_prober.go:28] interesting pod/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 11:38:54.471460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.471316    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 11:38:54.497296 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.497264    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b8260aa-6c78-4405-9a3e-b0db19a4216f" path="/var/lib/kubelet/pods/0b8260aa-6c78-4405-9a3e-b0db19a4216f/volumes"
Sep 07 11:38:54.497604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:54.497592    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2a18336-4d80-470a-aab8-2efa15611b00" path="/var/lib/kubelet/pods/a2a18336-4d80-470a-aab8-2efa15611b00/volumes"
Sep 07 11:38:55.056727 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:55.056698    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:55.057760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:55.057718    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:38:58.911305 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.911274    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:58.973863 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.973833    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-podinfo\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.973879    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8mmx\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-kube-api-access-s8mmx\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.973904    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-socket\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.973924    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-token\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.973946    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-envoy\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.973991    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-data\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974035    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-certs\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974102    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-credential-socket\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974129    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c2f135-d5c7-4ff2-ae57-8653af329a25-istiod-ca-cert\") pod \"31c2f135-d5c7-4ff2-ae57-8653af329a25\" (UID: \"31c2f135-d5c7-4ff2-ae57-8653af329a25\") "
Sep 07 11:38:58.974350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974194    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:58.974515 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974364    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:58.974515 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974376    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-data" (OuterVolumeSpecName: "istio-data") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:58.974619 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974517    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:58.974619 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974587    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:58.974619 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974610    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:58.974803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974624    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:58.974803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.974657    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c2f135-d5c7-4ff2-ae57-8653af329a25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:38:58.976134 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.976102    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-kube-api-access-s8mmx" (OuterVolumeSpecName: "kube-api-access-s8mmx") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "kube-api-access-s8mmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:58.976293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.976264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:38:58.976466 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.976447    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:38:58.976508 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:58.976464    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-token" (OuterVolumeSpecName: "istio-token") pod "31c2f135-d5c7-4ff2-ae57-8653af329a25" (UID: "31c2f135-d5c7-4ff2-ae57-8653af329a25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:38:59.075604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.075571    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:59.075604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.075599    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31c2f135-d5c7-4ff2-ae57-8653af329a25-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:59.075604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.075609    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:59.075868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.075617    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8mmx\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-kube-api-access-s8mmx\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:59.075868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.075625    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:59.075868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.075636    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31c2f135-d5c7-4ff2-ae57-8653af329a25-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:38:59.078888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.078857    2560 generic.go:356] "Generic (PLEG): container finished" podID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerID="aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d" exitCode=0
Sep 07 11:38:59.079020 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.078908    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" event={"ID":"31c2f135-d5c7-4ff2-ae57-8653af329a25","Type":"ContainerDied","Data":"aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d"}
Sep 07 11:38:59.079020 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.078919    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"
Sep 07 11:38:59.079020 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.078952    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj" event={"ID":"31c2f135-d5c7-4ff2-ae57-8653af329a25","Type":"ContainerDied","Data":"ad6c75edbe92db8731f238dac527e4f591d42e245c94f1c5533cc5075fce8d14"}
Sep 07 11:38:59.079020 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.078974    2560 scope.go:117] "RemoveContainer" containerID="aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d"
Sep 07 11:38:59.088577 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.088560    2560 scope.go:117] "RemoveContainer" containerID="12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"
Sep 07 11:38:59.099196 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.099173    2560 scope.go:117] "RemoveContainer" containerID="aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d"
Sep 07 11:38:59.099537 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:59.099512    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d\": container with ID starting with aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d not found: ID does not exist" containerID="aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d"
Sep 07 11:38:59.099630 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.099552    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d"} err="failed to get container status \"aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d\": rpc error: code = NotFound desc = could not find container \"aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d\": container with ID starting with aee21b44304ac045e9521396f5881bacf94f69bfe21401ccd759b9196e923a9d not found: ID does not exist"
Sep 07 11:38:59.099630 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.099577    2560 scope.go:117] "RemoveContainer" containerID="12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"
Sep 07 11:38:59.100518 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:38:59.100489    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835\": container with ID starting with 12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835 not found: ID does not exist" containerID="12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"
Sep 07 11:38:59.100664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.100642    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835"} err="failed to get container status \"12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835\": rpc error: code = NotFound desc = could not find container \"12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835\": container with ID starting with 12262ee5a757cb8cf3da9e7f5ba070f658e6a147cabf1d49d47d83fee8594835 not found: ID does not exist"
Sep 07 11:38:59.102929 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.102905    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"]
Sep 07 11:38:59.110793 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:38:59.110762    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3f8729d3-openshift-default-5f9f76477b-mr4rj"]
Sep 07 11:39:00.497349 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:00.497314    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" path="/var/lib/kubelet/pods/31c2f135-d5c7-4ff2-ae57-8653af329a25/volumes"
Sep 07 11:39:17.172418 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:17.172378    2560 generic.go:356] "Generic (PLEG): container finished" podID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerID="a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e" exitCode=0
Sep 07 11:39:17.172896 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:17.172452    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" event={"ID":"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a","Type":"ContainerDied","Data":"a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"}
Sep 07 11:39:17.172896 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:17.172864    2560 scope.go:117] "RemoveContainer" containerID="a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"
Sep 07 11:39:17.174004 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:17.173970    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:39:17.174118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:17.174065    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:39:17.174184 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:17.174116    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:39:18.179252 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:18.179220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" event={"ID":"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a","Type":"ContainerStarted","Data":"a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff"}
Sep 07 11:39:18.368289 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:18.368242    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:39:18.368926 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:18.368899    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 07 11:39:18.369027 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:18.368952    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 07 11:39:19.365126 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:19.365085    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 07 11:39:19.365523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:19.365180    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 07 11:39:20.365367 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:20.365331    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 07 11:39:20.365776 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:20.365395    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 07 11:39:21.364997 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:21.364957    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 07 11:39:21.365174 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:21.365023    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 07 11:39:22.364921 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:22.364883    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 07 11:39:22.365318 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:22.364942    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 07 11:39:23.368704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:23.368669    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:39:23.369194 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:23.369175    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:39:23.369971 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:23.369948    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:39:59.792994 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.792959    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"]
Sep 07 11:39:59.793395 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793381    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy"
Sep 07 11:39:59.793446 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793396    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy"
Sep 07 11:39:59.793446 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793410    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b8260aa-6c78-4405-9a3e-b0db19a4216f" containerName="payload-processing"
Sep 07 11:39:59.793446 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793416    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b8260aa-6c78-4405-9a3e-b0db19a4216f" containerName="payload-processing"
Sep 07 11:39:59.793539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793482    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b8260aa-6c78-4405-9a3e-b0db19a4216f" containerName="payload-processing"
Sep 07 11:39:59.793539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793492    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy"
Sep 07 11:39:59.793539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.793500    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy"
Sep 07 11:39:59.801046 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.801016    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:39:59.803550 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.803525    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:39:59.805107 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.805055    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"]
Sep 07 11:39:59.866840 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.866810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69806c4e-ca60-443e-aba5-584489a0a35a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl\" (UID: \"69806c4e-ca60-443e-aba5-584489a0a35a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:39:59.968018 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.967992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69806c4e-ca60-443e-aba5-584489a0a35a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl\" (UID: \"69806c4e-ca60-443e-aba5-584489a0a35a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:39:59.968564 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:39:59.968543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69806c4e-ca60-443e-aba5-584489a0a35a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl\" (UID: \"69806c4e-ca60-443e-aba5-584489a0a35a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:40:00.115146 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:00.115054    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:40:00.383377 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:00.383212    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"]
Sep 07 11:40:00.386152 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:40:00.386109    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69806c4e_ca60_443e_aba5_584489a0a35a.slice/crio-85928a756458c4b52956fcd0e33b7e6e853610d1f3f1480284037bcc4a0deca5 WatchSource:0}: Error finding container 85928a756458c4b52956fcd0e33b7e6e853610d1f3f1480284037bcc4a0deca5: Status 404 returned error can't find the container with id 85928a756458c4b52956fcd0e33b7e6e853610d1f3f1480284037bcc4a0deca5
Sep 07 11:40:00.423823 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:00.423781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl" event={"ID":"69806c4e-ca60-443e-aba5-584489a0a35a","Type":"ContainerStarted","Data":"85928a756458c4b52956fcd0e33b7e6e853610d1f3f1480284037bcc4a0deca5"}
Sep 07 11:40:01.430604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:01.430569    2560 generic.go:356] "Generic (PLEG): container finished" podID="69806c4e-ca60-443e-aba5-584489a0a35a" containerID="5d8d2a91dcbed74670ef96e81a563c767dcd055d2fe8b60c27637d9b79fec856" exitCode=0
Sep 07 11:40:01.431110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:01.430640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl" event={"ID":"69806c4e-ca60-443e-aba5-584489a0a35a","Type":"ContainerDied","Data":"5d8d2a91dcbed74670ef96e81a563c767dcd055d2fe8b60c27637d9b79fec856"}
Sep 07 11:40:02.577276 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:02.577252    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:40:02.695647 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:02.695530    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69806c4e-ca60-443e-aba5-584489a0a35a-maas-api-service-ca\") pod \"69806c4e-ca60-443e-aba5-584489a0a35a\" (UID: \"69806c4e-ca60-443e-aba5-584489a0a35a\") "
Sep 07 11:40:02.695937 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:02.695913    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69806c4e-ca60-443e-aba5-584489a0a35a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69806c4e-ca60-443e-aba5-584489a0a35a" (UID: "69806c4e-ca60-443e-aba5-584489a0a35a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:40:02.797395 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:02.797354    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69806c4e-ca60-443e-aba5-584489a0a35a-maas-api-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:03.442885 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:03.442852    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"
Sep 07 11:40:03.443086 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:03.442855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl" event={"ID":"69806c4e-ca60-443e-aba5-584489a0a35a","Type":"ContainerDied","Data":"85928a756458c4b52956fcd0e33b7e6e853610d1f3f1480284037bcc4a0deca5"}
Sep 07 11:40:03.443086 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:03.442985    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85928a756458c4b52956fcd0e33b7e6e853610d1f3f1480284037bcc4a0deca5"
Sep 07 11:40:10.540539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:10.540508    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"]
Sep 07 11:40:10.542955 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:10.542934    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bf198c-kqwcl"]
Sep 07 11:40:12.498445 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:12.498411    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69806c4e-ca60-443e-aba5-584489a0a35a" path="/var/lib/kubelet/pods/69806c4e-ca60-443e-aba5-584489a0a35a/volumes"
Sep 07 11:40:22.519937 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:22.519899    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"]
Sep 07 11:40:22.520565 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:22.520122    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" containerID="cri-o://a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff" gracePeriod=30
Sep 07 11:40:23.277181 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.277146    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"]
Sep 07 11:40:23.277718 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.277700    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69806c4e-ca60-443e-aba5-584489a0a35a" containerName="revoke-keys"
Sep 07 11:40:23.277718 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.277719    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69806c4e-ca60-443e-aba5-584489a0a35a" containerName="revoke-keys"
Sep 07 11:40:23.277856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.277765    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy"
Sep 07 11:40:23.277856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.277775    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c2f135-d5c7-4ff2-ae57-8653af329a25" containerName="istio-proxy"
Sep 07 11:40:23.277856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.277842    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69806c4e-ca60-443e-aba5-584489a0a35a" containerName="revoke-keys"
Sep 07 11:40:23.284425 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.282483    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.286835 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.286810    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-f5267c-openshift-default-dockercfg-szjfp\""
Sep 07 11:40:23.292877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.292856    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"]
Sep 07 11:40:23.373075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.373026    2560 patch_prober.go:28] interesting pod/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 11:40:23.373273 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.373127    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 11:40:23.388014 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.387962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-credential-socket\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388084    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-podinfo\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388157    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-data\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388375 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-token\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388375 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-envoy\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388375 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2031c73c-4023-474a-8704-35ab22eb8f3b-istiod-ca-cert\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388375 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-socket\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388375 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388370    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx9x8\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-kube-api-access-cx9x8\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.388638 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.388417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-certs\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489438 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489401    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-credential-socket\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489455    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-podinfo\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489498    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-data\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-token\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489572    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-envoy\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2031c73c-4023-474a-8704-35ab22eb8f3b-istiod-ca-cert\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489633 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-socket\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9x8\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-kube-api-access-cx9x8\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489725    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-certs\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489875    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-credential-socket\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.489995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.489931    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-data\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.490192 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.490084    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-certs\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.490192 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.490127    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-socket\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.490668 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.490638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2031c73c-4023-474a-8704-35ab22eb8f3b-istiod-ca-cert\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.492159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.492139    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-envoy\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.492262 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.492227    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-podinfo\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.497985 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.497964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-token\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.498075 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.498035    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx9x8\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-kube-api-access-cx9x8\") pod \"e2e-noten-f5267c-openshift-default-64f7468fd-4w6th\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") " pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.596523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.596440    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:23.869662 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.869636    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"]
Sep 07 11:40:23.871670 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:40:23.871642    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2031c73c_4023_474a_8704_35ab22eb8f3b.slice/crio-5736c40417d1ff6d1898eefb452502d7380732dfcb5a376c908aa26c6cc23420 WatchSource:0}: Error finding container 5736c40417d1ff6d1898eefb452502d7380732dfcb5a376c908aa26c6cc23420: Status 404 returned error can't find the container with id 5736c40417d1ff6d1898eefb452502d7380732dfcb5a376c908aa26c6cc23420
Sep 07 11:40:23.873927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.873891    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:40:23.874015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.873977    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:40:23.874015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:23.874011    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:40:24.560089 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:24.560056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th" event={"ID":"2031c73c-4023-474a-8704-35ab22eb8f3b","Type":"ContainerStarted","Data":"88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35"}
Sep 07 11:40:24.560250 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:24.560114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th" event={"ID":"2031c73c-4023-474a-8704-35ab22eb8f3b","Type":"ContainerStarted","Data":"5736c40417d1ff6d1898eefb452502d7380732dfcb5a376c908aa26c6cc23420"}
Sep 07 11:40:24.580908 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:24.580783    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th" podStartSLOduration=1.58076632 podStartE2EDuration="1.58076632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:40:24.580036212 +0000 UTC m=+1056.527386996" watchObservedRunningTime="2026-09-07 11:40:24.58076632 +0000 UTC m=+1056.528117096"
Sep 07 11:40:24.597473 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:24.597445    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:24.602828 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:24.602807    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:25.566136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:25.566102    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:25.567210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:25.567183    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:27.779497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.779463    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:40:27.833630 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833543    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-podinfo\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.833630 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833592    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-data\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.833871 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833634    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-certs\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.833871 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833729    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istiod-ca-cert\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.833871 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833800    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-envoy\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.834044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833955    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:27.834044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833966    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:27.834044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.833963    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-token\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.834200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834075    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzrrf\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-kube-api-access-nzrrf\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.834200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834117    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-socket\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.834200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834118    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:40:27.834363 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834228    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-credential-socket\") pod \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\" (UID: \"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a\") "
Sep 07 11:40:27.834496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834467    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:27.834662 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834634    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.834758 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834668    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.834758 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834681    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.834758 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834676    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:27.834758 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.834693    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.835854 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.835824    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:40:27.836370 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.836340    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:40:27.836474 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.836407    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:27.836719 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.836703    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-kube-api-access-nzrrf" (OuterVolumeSpecName: "kube-api-access-nzrrf") pod "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" (UID: "520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a"). InnerVolumeSpecName "kube-api-access-nzrrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:40:27.936150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.936112    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.936150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.936144    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.936150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.936154    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.936150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.936161    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:27.936429 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:27.936170    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzrrf\" (UniqueName: \"kubernetes.io/projected/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a-kube-api-access-nzrrf\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:28.583039 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.583008    2560 generic.go:356] "Generic (PLEG): container finished" podID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerID="a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff" exitCode=0
Sep 07 11:40:28.583227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.583055    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"
Sep 07 11:40:28.583227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.583097    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" event={"ID":"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a","Type":"ContainerDied","Data":"a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff"}
Sep 07 11:40:28.583227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.583147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm" event={"ID":"520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a","Type":"ContainerDied","Data":"9ad6856a1e4b0725cf28c2410a6cb6b21f2cd96867fdda0b2171c94ad0d6b007"}
Sep 07 11:40:28.583227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.583169    2560 scope.go:117] "RemoveContainer" containerID="a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff"
Sep 07 11:40:28.591976 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.591957    2560 scope.go:117] "RemoveContainer" containerID="a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"
Sep 07 11:40:28.600090 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.600071    2560 scope.go:117] "RemoveContainer" containerID="a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff"
Sep 07 11:40:28.600441 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:40:28.600412    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff\": container with ID starting with a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff not found: ID does not exist" containerID="a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff"
Sep 07 11:40:28.600539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.600462    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff"} err="failed to get container status \"a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff\": rpc error: code = NotFound desc = could not find container \"a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff\": container with ID starting with a1c2153a26773827205d4165df353a9140689435e1c969a019ee1852af7142ff not found: ID does not exist"
Sep 07 11:40:28.600539 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.600481    2560 scope.go:117] "RemoveContainer" containerID="a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"
Sep 07 11:40:28.600876 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:40:28.600851    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e\": container with ID starting with a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e not found: ID does not exist" containerID="a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"
Sep 07 11:40:28.600941 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.600886    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e"} err="failed to get container status \"a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e\": rpc error: code = NotFound desc = could not find container \"a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e\": container with ID starting with a05e9076c204f10ed0d4fe43dd423291ae389543f2ee1d29dfc1c7b8ed9d306e not found: ID does not exist"
Sep 07 11:40:28.602220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.602202    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"]
Sep 07 11:40:28.605867 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:28.605845    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-bf198c-openshift-default-7f4b8c6764-qhddm"]
Sep 07 11:40:30.497366 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:30.497332    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" path="/var/lib/kubelet/pods/520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a/volumes"
Sep 07 11:40:36.464627 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.464593    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"]
Sep 07 11:40:36.465085 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.465069    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy"
Sep 07 11:40:36.465132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.465086    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy"
Sep 07 11:40:36.465170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.465147    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy"
Sep 07 11:40:36.465170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.465156    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy"
Sep 07 11:40:36.469621 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.469601    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.471730 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.471708    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d6b326a7-dockercfg-dbm2k\""
Sep 07 11:40:36.471850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.471708    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d6b326a7\""
Sep 07 11:40:36.476036 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.476010    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"]
Sep 07 11:40:36.498726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.498699    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"]
Sep 07 11:40:36.499100 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.499087    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy"
Sep 07 11:40:36.499143 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.499102    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="520f0ab4-c0d9-4d1b-b7ae-d80eaac7be5a" containerName="istio-proxy"
Sep 07 11:40:36.502466 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.502442    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.512133 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.512107    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"]
Sep 07 11:40:36.516901 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.516877    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.517035 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.517013    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b5kp\" (UniqueName: \"kubernetes.io/projected/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-kube-api-access-9b5kp\") pod \"payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.618220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.618184    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs4j2\" (UniqueName: \"kubernetes.io/projected/108ad278-6cee-4e02-abd4-609e40a9c53f-kube-api-access-rs4j2\") pod \"payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") " pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.618410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.618277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/108ad278-6cee-4e02-abd4-609e40a9c53f-plugins-config-volume\") pod \"payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") " pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.618410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.618298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b5kp\" (UniqueName: \"kubernetes.io/projected/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-kube-api-access-9b5kp\") pod \"payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.618410 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.618343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.618966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.618947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.626295 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.626273    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b5kp\" (UniqueName: \"kubernetes.io/projected/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-kube-api-access-9b5kp\") pod \"payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.719332 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.719240    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs4j2\" (UniqueName: \"kubernetes.io/projected/108ad278-6cee-4e02-abd4-609e40a9c53f-kube-api-access-rs4j2\") pod \"payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") " pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.719332 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.719330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/108ad278-6cee-4e02-abd4-609e40a9c53f-plugins-config-volume\") pod \"payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") " pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.720034 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.720014    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/108ad278-6cee-4e02-abd4-609e40a9c53f-plugins-config-volume\") pod \"payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") " pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.727928 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.727910    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs4j2\" (UniqueName: \"kubernetes.io/projected/108ad278-6cee-4e02-abd4-609e40a9c53f-kube-api-access-rs4j2\") pod \"payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") " pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:36.780433 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.780408    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:36.813213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:36.813184    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:37.054341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.054312    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"]
Sep 07 11:40:37.056281 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:40:37.056256    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb35e068_f1c4_4d2c_8bc7_c1f31eb1d776.slice/crio-bf4c86df3f4d503b40ab09b3663fc622e1a17b5a31b060fb064fd65dc1ca15ed WatchSource:0}: Error finding container bf4c86df3f4d503b40ab09b3663fc622e1a17b5a31b060fb064fd65dc1ca15ed: Status 404 returned error can't find the container with id bf4c86df3f4d503b40ab09b3663fc622e1a17b5a31b060fb064fd65dc1ca15ed
Sep 07 11:40:37.108777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.108267    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"]
Sep 07 11:40:37.112722 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:40:37.112571    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108ad278_6cee_4e02_abd4_609e40a9c53f.slice/crio-3d759f1486dc701f2c41e3b767fc471ea68760402fdcbf17b536d4abf9eb65d2 WatchSource:0}: Error finding container 3d759f1486dc701f2c41e3b767fc471ea68760402fdcbf17b536d4abf9eb65d2: Status 404 returned error can't find the container with id 3d759f1486dc701f2c41e3b767fc471ea68760402fdcbf17b536d4abf9eb65d2
Sep 07 11:40:37.632975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.632933    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh" event={"ID":"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776","Type":"ContainerStarted","Data":"5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55"}
Sep 07 11:40:37.633421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.632999    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh" event={"ID":"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776","Type":"ContainerStarted","Data":"bf4c86df3f4d503b40ab09b3663fc622e1a17b5a31b060fb064fd65dc1ca15ed"}
Sep 07 11:40:37.633421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.633027    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:37.634664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.634639    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5" event={"ID":"108ad278-6cee-4e02-abd4-609e40a9c53f","Type":"ContainerStarted","Data":"7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534"}
Sep 07 11:40:37.634817 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.634682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5" event={"ID":"108ad278-6cee-4e02-abd4-609e40a9c53f","Type":"ContainerStarted","Data":"3d759f1486dc701f2c41e3b767fc471ea68760402fdcbf17b536d4abf9eb65d2"}
Sep 07 11:40:37.634817 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.634791    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:37.652310 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.652242    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh" podStartSLOduration=1.652223478 podStartE2EDuration="1.652223478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:40:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:40:37.650668366 +0000 UTC m=+1069.598019152" watchObservedRunningTime="2026-09-07 11:40:37.652223478 +0000 UTC m=+1069.599574261"
Sep 07 11:40:37.670918 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:37.670872    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5" podStartSLOduration=1.670855572 podStartE2EDuration="1.670855572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:40:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:40:37.669192647 +0000 UTC m=+1069.616543426" watchObservedRunningTime="2026-09-07 11:40:37.670855572 +0000 UTC m=+1069.618206360"
Sep 07 11:40:40.990511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:40.990422    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"]
Sep 07 11:40:40.990993 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:40.990677    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th" podUID="2031c73c-4023-474a-8704-35ab22eb8f3b" containerName="istio-proxy" containerID="cri-o://88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35" gracePeriod=30
Sep 07 11:40:46.254533 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.254465    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:46.408534 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408492    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2031c73c-4023-474a-8704-35ab22eb8f3b-istiod-ca-cert\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.408692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408546    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-envoy\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.408692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408594    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-socket\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.408851 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408690    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx9x8\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-kube-api-access-cx9x8\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.408851 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408800    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-certs\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.408851 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408832    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-data\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.409015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408864    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-credential-socket\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.409015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408901    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-token\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.409015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408956    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-podinfo\") pod \"2031c73c-4023-474a-8704-35ab22eb8f3b\" (UID: \"2031c73c-4023-474a-8704-35ab22eb8f3b\") "
Sep 07 11:40:46.409015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.408988    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2031c73c-4023-474a-8704-35ab22eb8f3b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:40:46.409243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409101    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:46.409243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409113    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:46.409243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409150    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:46.409243 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409183    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:46.409458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409371    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.409458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409390    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.409458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409402    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.409458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409415    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2031c73c-4023-474a-8704-35ab22eb8f3b-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.409458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.409428    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.411050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.411020    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-kube-api-access-cx9x8" (OuterVolumeSpecName: "kube-api-access-cx9x8") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "kube-api-access-cx9x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:40:46.411180 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.411105    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:40:46.411293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.411265    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:40:46.411402 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.411331    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2031c73c-4023-474a-8704-35ab22eb8f3b" (UID: "2031c73c-4023-474a-8704-35ab22eb8f3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:40:46.510345 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.510312    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.510345 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.510341    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx9x8\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-kube-api-access-cx9x8\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.510511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.510355    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.510511 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.510367    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2031c73c-4023-474a-8704-35ab22eb8f3b-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:46.683356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.683315    2560 generic.go:356] "Generic (PLEG): container finished" podID="2031c73c-4023-474a-8704-35ab22eb8f3b" containerID="88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35" exitCode=0
Sep 07 11:40:46.683526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.683375    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"
Sep 07 11:40:46.683526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.683397    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th" event={"ID":"2031c73c-4023-474a-8704-35ab22eb8f3b","Type":"ContainerDied","Data":"88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35"}
Sep 07 11:40:46.683526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.683460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th" event={"ID":"2031c73c-4023-474a-8704-35ab22eb8f3b","Type":"ContainerDied","Data":"5736c40417d1ff6d1898eefb452502d7380732dfcb5a376c908aa26c6cc23420"}
Sep 07 11:40:46.683526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.683485    2560 scope.go:117] "RemoveContainer" containerID="88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35"
Sep 07 11:40:46.692569 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.692550    2560 scope.go:117] "RemoveContainer" containerID="88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35"
Sep 07 11:40:46.692890 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:40:46.692871    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35\": container with ID starting with 88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35 not found: ID does not exist" containerID="88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35"
Sep 07 11:40:46.692957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.692899    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35"} err="failed to get container status \"88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35\": rpc error: code = NotFound desc = could not find container \"88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35\": container with ID starting with 88777f4cafe737dd7f2c08a454a2a9bbac956a0ffacd4fd32fb65c3345b61d35 not found: ID does not exist"
Sep 07 11:40:46.699804 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.699781    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"]
Sep 07 11:40:46.707136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:46.707114    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-f5267c-openshift-default-64f7468fd-4w6th"]
Sep 07 11:40:48.498882 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:48.498849    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2031c73c-4023-474a-8704-35ab22eb8f3b" path="/var/lib/kubelet/pods/2031c73c-4023-474a-8704-35ab22eb8f3b/volumes"
Sep 07 11:40:48.642314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:48.642287    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:40:48.642466 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:48.642369    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:40:52.425490 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.425457    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"]
Sep 07 11:40:52.425945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.425813    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq" podUID="a66910d5-fbae-4c1f-bac3-eca690e3fdec" containerName="payload-processing" containerID="cri-o://0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f" gracePeriod=30
Sep 07 11:40:52.433069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.433043    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"]
Sep 07 11:40:52.433276 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.433254    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw" podUID="7a82662b-3dca-4adb-9e6d-2777a4e2d49b" containerName="payload-pre-processing" containerID="cri-o://66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4" gracePeriod=30
Sep 07 11:40:52.695604 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.695571    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:40:52.720297 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.720271    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:40:52.721645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.721614    2560 generic.go:356] "Generic (PLEG): container finished" podID="a66910d5-fbae-4c1f-bac3-eca690e3fdec" containerID="0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f" exitCode=0
Sep 07 11:40:52.721803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.721653    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"
Sep 07 11:40:52.721803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.721674    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq" event={"ID":"a66910d5-fbae-4c1f-bac3-eca690e3fdec","Type":"ContainerDied","Data":"0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f"}
Sep 07 11:40:52.721803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.721729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq" event={"ID":"a66910d5-fbae-4c1f-bac3-eca690e3fdec","Type":"ContainerDied","Data":"e143c28c8aa9188fc09ed060fe1c8b4e58f146c9be71234917fa508511be6451"}
Sep 07 11:40:52.721803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.721767    2560 scope.go:117] "RemoveContainer" containerID="0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f"
Sep 07 11:40:52.723193 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.723170    2560 generic.go:356] "Generic (PLEG): container finished" podID="7a82662b-3dca-4adb-9e6d-2777a4e2d49b" containerID="66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4" exitCode=0
Sep 07 11:40:52.723280 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.723212    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"
Sep 07 11:40:52.723280 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.723256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw" event={"ID":"7a82662b-3dca-4adb-9e6d-2777a4e2d49b","Type":"ContainerDied","Data":"66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4"}
Sep 07 11:40:52.723397 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.723310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw" event={"ID":"7a82662b-3dca-4adb-9e6d-2777a4e2d49b","Type":"ContainerDied","Data":"dd07bd5edff9a7b55c6fd39cfbabcbf3b422446546da6ad1e8282394d89f3172"}
Sep 07 11:40:52.730749 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.730722    2560 scope.go:117] "RemoveContainer" containerID="0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f"
Sep 07 11:40:52.731040 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:40:52.731012    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f\": container with ID starting with 0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f not found: ID does not exist" containerID="0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f"
Sep 07 11:40:52.731105 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.731051    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f"} err="failed to get container status \"0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f\": rpc error: code = NotFound desc = could not find container \"0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f\": container with ID starting with 0cac049d897c5f91e15d2adba6066bdcae64eb4c71d2a65108d2ef65a8df063f not found: ID does not exist"
Sep 07 11:40:52.731105 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.731080    2560 scope.go:117] "RemoveContainer" containerID="66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4"
Sep 07 11:40:52.740109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.740076    2560 scope.go:117] "RemoveContainer" containerID="66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4"
Sep 07 11:40:52.740436 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:40:52.740414    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4\": container with ID starting with 66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4 not found: ID does not exist" containerID="66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4"
Sep 07 11:40:52.740535 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.740442    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4"} err="failed to get container status \"66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4\": rpc error: code = NotFound desc = could not find container \"66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4\": container with ID starting with 66361209be7bd4a3d8fe2c6c867b4ef8d5f875f2058bc7651174d42c0e2dc9f4 not found: ID does not exist"
Sep 07 11:40:52.872629 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.872593    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsczt\" (UniqueName: \"kubernetes.io/projected/a66910d5-fbae-4c1f-bac3-eca690e3fdec-kube-api-access-tsczt\") pod \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") "
Sep 07 11:40:52.872848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.872633    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66910d5-fbae-4c1f-bac3-eca690e3fdec-plugins-config-volume\") pod \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\" (UID: \"a66910d5-fbae-4c1f-bac3-eca690e3fdec\") "
Sep 07 11:40:52.872848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.872719    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-plugins-config-volume\") pod \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") "
Sep 07 11:40:52.872848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.872819    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcblp\" (UniqueName: \"kubernetes.io/projected/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-kube-api-access-fcblp\") pod \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\" (UID: \"7a82662b-3dca-4adb-9e6d-2777a4e2d49b\") "
Sep 07 11:40:52.873128 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.873102    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a82662b-3dca-4adb-9e6d-2777a4e2d49b" (UID: "7a82662b-3dca-4adb-9e6d-2777a4e2d49b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:40:52.873171 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.873100    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66910d5-fbae-4c1f-bac3-eca690e3fdec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a66910d5-fbae-4c1f-bac3-eca690e3fdec" (UID: "a66910d5-fbae-4c1f-bac3-eca690e3fdec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:40:52.874832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.874810    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-kube-api-access-fcblp" (OuterVolumeSpecName: "kube-api-access-fcblp") pod "7a82662b-3dca-4adb-9e6d-2777a4e2d49b" (UID: "7a82662b-3dca-4adb-9e6d-2777a4e2d49b"). InnerVolumeSpecName "kube-api-access-fcblp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:40:52.874904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.874887    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66910d5-fbae-4c1f-bac3-eca690e3fdec-kube-api-access-tsczt" (OuterVolumeSpecName: "kube-api-access-tsczt") pod "a66910d5-fbae-4c1f-bac3-eca690e3fdec" (UID: "a66910d5-fbae-4c1f-bac3-eca690e3fdec"). InnerVolumeSpecName "kube-api-access-tsczt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:40:52.973861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.973788    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsczt\" (UniqueName: \"kubernetes.io/projected/a66910d5-fbae-4c1f-bac3-eca690e3fdec-kube-api-access-tsczt\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:52.973861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.973815    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66910d5-fbae-4c1f-bac3-eca690e3fdec-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:52.973861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.973826    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:52.973861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:52.973834    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcblp\" (UniqueName: \"kubernetes.io/projected/7a82662b-3dca-4adb-9e6d-2777a4e2d49b-kube-api-access-fcblp\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:40:53.046773 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:53.046733    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"]
Sep 07 11:40:53.050161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:53.050136    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b7e4c1-7fd68d75db-46dnw"]
Sep 07 11:40:53.060166 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:53.060140    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"]
Sep 07 11:40:53.063074 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:53.063046    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b7e4c1-df7dc7cc-9lwvq"]
Sep 07 11:40:54.497728 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:54.497694    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a82662b-3dca-4adb-9e6d-2777a4e2d49b" path="/var/lib/kubelet/pods/7a82662b-3dca-4adb-9e6d-2777a4e2d49b/volumes"
Sep 07 11:40:54.498193 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:54.498077    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66910d5-fbae-4c1f-bac3-eca690e3fdec" path="/var/lib/kubelet/pods/a66910d5-fbae-4c1f-bac3-eca690e3fdec/volumes"
Sep 07 11:40:59.564043 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:59.564011    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"]
Sep 07 11:40:59.564438 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:40:59.564287    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy" containerID="cri-o://9a0aa6ffd3e39f5b25240bef3babdc1d0e7d86fa19a717dbf05ffd6fee4412f4" gracePeriod=30
Sep 07 11:41:04.786472 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.786434    2560 generic.go:356] "Generic (PLEG): container finished" podID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerID="9a0aa6ffd3e39f5b25240bef3babdc1d0e7d86fa19a717dbf05ffd6fee4412f4" exitCode=0
Sep 07 11:41:04.786912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.786498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" event={"ID":"e5787665-50f7-40a0-b9b7-0cbdc44ccf58","Type":"ContainerDied","Data":"9a0aa6ffd3e39f5b25240bef3babdc1d0e7d86fa19a717dbf05ffd6fee4412f4"}
Sep 07 11:41:04.786912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.786563    2560 scope.go:117] "RemoveContainer" containerID="20afd7226c33644e2c00e0b5dc5930bcc6267528af8a345514735e443ec4e81c"
Sep 07 11:41:04.814041 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.814021    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:41:04.883383 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883307    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-certs\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883383 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883355    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-token\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883416    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-socket\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883458    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-envoy\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883521    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-credential-socket\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883552    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-742nf\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-kube-api-access-742nf\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883581    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-data\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883612    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:41:04.883694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883616    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-podinfo\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.884054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883728    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:41:04.884054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883822    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istiod-ca-cert\") pod \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\" (UID: \"e5787665-50f7-40a0-b9b7-0cbdc44ccf58\") "
Sep 07 11:41:04.884054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883818    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:41:04.884054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.883872    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:41:04.884240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.884164    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:41:04.884240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.884183    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.884240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.884205    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.884240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.884220    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.884240 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.884232    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.885689 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.885664    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:41:04.885821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.885670    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:41:04.886078 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.886056    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:41:04.886149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.886056    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-kube-api-access-742nf" (OuterVolumeSpecName: "kube-api-access-742nf") pod "e5787665-50f7-40a0-b9b7-0cbdc44ccf58" (UID: "e5787665-50f7-40a0-b9b7-0cbdc44ccf58"). InnerVolumeSpecName "kube-api-access-742nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:41:04.985204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.985167    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-742nf\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-kube-api-access-742nf\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.985204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.985198    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.985204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.985209    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.985204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.985218    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:04.985465 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:04.985226    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5787665-50f7-40a0-b9b7-0cbdc44ccf58-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:05.793121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:05.793091    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"
Sep 07 11:41:05.793504 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:05.793097    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx" event={"ID":"e5787665-50f7-40a0-b9b7-0cbdc44ccf58","Type":"ContainerDied","Data":"b04c6d9382558f4eac18ee7d4f35ffef7d81ed0d4eaca3b306e33ce5642b222a"}
Sep 07 11:41:05.793504 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:05.793236    2560 scope.go:117] "RemoveContainer" containerID="9a0aa6ffd3e39f5b25240bef3babdc1d0e7d86fa19a717dbf05ffd6fee4412f4"
Sep 07 11:41:05.830340 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:05.830314    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"]
Sep 07 11:41:05.834703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:05.834678    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b7e4c1-openshift-default-5d45774946-dfsrx"]
Sep 07 11:41:06.497518 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:06.497481    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" path="/var/lib/kubelet/pods/e5787665-50f7-40a0-b9b7-0cbdc44ccf58/volumes"
Sep 07 11:41:07.070912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.070869    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"]
Sep 07 11:41:07.071297 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071286    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy"
Sep 07 11:41:07.071352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071301    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy"
Sep 07 11:41:07.071352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071327    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66910d5-fbae-4c1f-bac3-eca690e3fdec" containerName="payload-processing"
Sep 07 11:41:07.071352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071334    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66910d5-fbae-4c1f-bac3-eca690e3fdec" containerName="payload-processing"
Sep 07 11:41:07.071352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071344    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a82662b-3dca-4adb-9e6d-2777a4e2d49b" containerName="payload-pre-processing"
Sep 07 11:41:07.071352 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071350    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a82662b-3dca-4adb-9e6d-2777a4e2d49b" containerName="payload-pre-processing"
Sep 07 11:41:07.071527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071359    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2031c73c-4023-474a-8704-35ab22eb8f3b" containerName="istio-proxy"
Sep 07 11:41:07.071527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071364    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2031c73c-4023-474a-8704-35ab22eb8f3b" containerName="istio-proxy"
Sep 07 11:41:07.071527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071432    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a82662b-3dca-4adb-9e6d-2777a4e2d49b" containerName="payload-pre-processing"
Sep 07 11:41:07.071527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071440    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy"
Sep 07 11:41:07.071527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071447    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2031c73c-4023-474a-8704-35ab22eb8f3b" containerName="istio-proxy"
Sep 07 11:41:07.071527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.071455    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66910d5-fbae-4c1f-bac3-eca690e3fdec" containerName="payload-processing"
Sep 07 11:41:07.075991 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.075974    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:07.077881 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.077862    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:41:07.080888 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.080862    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"]
Sep 07 11:41:07.102335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.102307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/580a19d9-0450-4a37-a615-93fb8014fb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4\" (UID: \"580a19d9-0450-4a37-a615-93fb8014fb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:07.203523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.203493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/580a19d9-0450-4a37-a615-93fb8014fb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4\" (UID: \"580a19d9-0450-4a37-a615-93fb8014fb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:07.204177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.204155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/580a19d9-0450-4a37-a615-93fb8014fb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4\" (UID: \"580a19d9-0450-4a37-a615-93fb8014fb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:07.388909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.388833    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:07.659195 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.659169    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"]
Sep 07 11:41:07.661503 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:41:07.661469    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod580a19d9_0450_4a37_a615_93fb8014fb81.slice/crio-c1be54437ff13462758d0998f5a82772dcbbced941d616ed407f836b8ae00644 WatchSource:0}: Error finding container c1be54437ff13462758d0998f5a82772dcbbced941d616ed407f836b8ae00644: Status 404 returned error can't find the container with id c1be54437ff13462758d0998f5a82772dcbbced941d616ed407f836b8ae00644
Sep 07 11:41:07.804964 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:07.804931    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4" event={"ID":"580a19d9-0450-4a37-a615-93fb8014fb81","Type":"ContainerStarted","Data":"c1be54437ff13462758d0998f5a82772dcbbced941d616ed407f836b8ae00644"}
Sep 07 11:41:08.810984 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:08.810948    2560 generic.go:356] "Generic (PLEG): container finished" podID="580a19d9-0450-4a37-a615-93fb8014fb81" containerID="2ae8ebba46144da2665f7d1a05a7f445604303ce26703b411280be3efb2a1801" exitCode=0
Sep 07 11:41:08.811335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:08.811029    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4" event={"ID":"580a19d9-0450-4a37-a615-93fb8014fb81","Type":"ContainerDied","Data":"2ae8ebba46144da2665f7d1a05a7f445604303ce26703b411280be3efb2a1801"}
Sep 07 11:41:09.946670 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:09.946645    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:10.026664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.026625    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/580a19d9-0450-4a37-a615-93fb8014fb81-maas-api-service-ca\") pod \"580a19d9-0450-4a37-a615-93fb8014fb81\" (UID: \"580a19d9-0450-4a37-a615-93fb8014fb81\") "
Sep 07 11:41:10.027024 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.027003    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/580a19d9-0450-4a37-a615-93fb8014fb81-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "580a19d9-0450-4a37-a615-93fb8014fb81" (UID: "580a19d9-0450-4a37-a615-93fb8014fb81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:41:10.127598 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.127524    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/580a19d9-0450-4a37-a615-93fb8014fb81-maas-api-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:10.714162 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.714123    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"]
Sep 07 11:41:10.714417 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.714386    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7" podUID="3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" containerName="payload-processing" containerID="cri-o://31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534" gracePeriod=30
Sep 07 11:41:10.822645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.822570    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"
Sep 07 11:41:10.822645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.822603    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4" event={"ID":"580a19d9-0450-4a37-a615-93fb8014fb81","Type":"ContainerDied","Data":"c1be54437ff13462758d0998f5a82772dcbbced941d616ed407f836b8ae00644"}
Sep 07 11:41:10.822854 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.822652    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1be54437ff13462758d0998f5a82772dcbbced941d616ed407f836b8ae00644"
Sep 07 11:41:10.980271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:10.980246    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:41:11.035709 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.035678    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5vhl\" (UniqueName: \"kubernetes.io/projected/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-kube-api-access-q5vhl\") pod \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") "
Sep 07 11:41:11.035869 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.035786    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-plugins-config-volume\") pod \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\" (UID: \"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28\") "
Sep 07 11:41:11.036212 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.036182    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" (UID: "3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:41:11.037971 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.037940    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-kube-api-access-q5vhl" (OuterVolumeSpecName: "kube-api-access-q5vhl") pod "3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" (UID: "3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28"). InnerVolumeSpecName "kube-api-access-q5vhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:41:11.136966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.136856    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5vhl\" (UniqueName: \"kubernetes.io/projected/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-kube-api-access-q5vhl\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:11.136966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.136891    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:11.830615 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.830578    2560 generic.go:356] "Generic (PLEG): container finished" podID="3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" containerID="31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534" exitCode=0
Sep 07 11:41:11.830836 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.830635    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"
Sep 07 11:41:11.830836 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.830656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7" event={"ID":"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28","Type":"ContainerDied","Data":"31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534"}
Sep 07 11:41:11.830836 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.830756    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7" event={"ID":"3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28","Type":"ContainerDied","Data":"754ec1a0b7812edec1a715b6fa678f82099f5082300613d72ede955725e450a0"}
Sep 07 11:41:11.830836 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.830781    2560 scope.go:117] "RemoveContainer" containerID="31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534"
Sep 07 11:41:11.841301 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.841116    2560 scope.go:117] "RemoveContainer" containerID="31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534"
Sep 07 11:41:11.841440 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:41:11.841415    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534\": container with ID starting with 31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534 not found: ID does not exist" containerID="31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534"
Sep 07 11:41:11.841502 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.841448    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534"} err="failed to get container status \"31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534\": rpc error: code = NotFound desc = could not find container \"31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534\": container with ID starting with 31842ac3c544c551bc95dc09c28b2531542646cc76b61620da1763e90517c534 not found: ID does not exist"
Sep 07 11:41:11.851351 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.851325    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"]
Sep 07 11:41:11.858215 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:11.858192    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef61c5-5c9f57d88c-n2rj7"]
Sep 07 11:41:12.497726 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:12.497689    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" path="/var/lib/kubelet/pods/3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28/volumes"
Sep 07 11:41:17.996718 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:17.996683    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"]
Sep 07 11:41:17.997115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:17.997029    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5" podUID="108ad278-6cee-4e02-abd4-609e40a9c53f" containerName="payload-processing" containerID="cri-o://7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534" gracePeriod=30
Sep 07 11:41:18.006386 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.006356    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"]
Sep 07 11:41:18.006622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.006595    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh" podUID="db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" containerName="payload-pre-processing" containerID="cri-o://5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55" gracePeriod=30
Sep 07 11:41:18.163284 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.163242    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"]
Sep 07 11:41:18.166464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.166407    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6b326a7-ghqg4"]
Sep 07 11:41:18.273679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.273650    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:41:18.276637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.276619    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:41:18.409159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.409121    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-plugins-config-volume\") pod \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") "
Sep 07 11:41:18.409331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.409193    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/108ad278-6cee-4e02-abd4-609e40a9c53f-plugins-config-volume\") pod \"108ad278-6cee-4e02-abd4-609e40a9c53f\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") "
Sep 07 11:41:18.409331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.409229    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b5kp\" (UniqueName: \"kubernetes.io/projected/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-kube-api-access-9b5kp\") pod \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\" (UID: \"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776\") "
Sep 07 11:41:18.409331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.409272    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs4j2\" (UniqueName: \"kubernetes.io/projected/108ad278-6cee-4e02-abd4-609e40a9c53f-kube-api-access-rs4j2\") pod \"108ad278-6cee-4e02-abd4-609e40a9c53f\" (UID: \"108ad278-6cee-4e02-abd4-609e40a9c53f\") "
Sep 07 11:41:18.409573 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.409544    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" (UID: "db35e068-f1c4-4d2c-8bc7-c1f31eb1d776"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:41:18.409621 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.409601    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/108ad278-6cee-4e02-abd4-609e40a9c53f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "108ad278-6cee-4e02-abd4-609e40a9c53f" (UID: "108ad278-6cee-4e02-abd4-609e40a9c53f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:41:18.411498 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.411467    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-kube-api-access-9b5kp" (OuterVolumeSpecName: "kube-api-access-9b5kp") pod "db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" (UID: "db35e068-f1c4-4d2c-8bc7-c1f31eb1d776"). InnerVolumeSpecName "kube-api-access-9b5kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:41:18.411618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.411504    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/108ad278-6cee-4e02-abd4-609e40a9c53f-kube-api-access-rs4j2" (OuterVolumeSpecName: "kube-api-access-rs4j2") pod "108ad278-6cee-4e02-abd4-609e40a9c53f" (UID: "108ad278-6cee-4e02-abd4-609e40a9c53f"). InnerVolumeSpecName "kube-api-access-rs4j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:41:18.497804 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.497768    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="580a19d9-0450-4a37-a615-93fb8014fb81" path="/var/lib/kubelet/pods/580a19d9-0450-4a37-a615-93fb8014fb81/volumes"
Sep 07 11:41:18.510093 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.510071    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs4j2\" (UniqueName: \"kubernetes.io/projected/108ad278-6cee-4e02-abd4-609e40a9c53f-kube-api-access-rs4j2\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:18.510093 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.510093    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:18.510220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.510104    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/108ad278-6cee-4e02-abd4-609e40a9c53f-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:18.510220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.510113    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b5kp\" (UniqueName: \"kubernetes.io/projected/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776-kube-api-access-9b5kp\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:41:18.696119 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696088    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"]
Sep 07 11:41:18.696497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696482    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="108ad278-6cee-4e02-abd4-609e40a9c53f" containerName="payload-processing"
Sep 07 11:41:18.696542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696500    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="108ad278-6cee-4e02-abd4-609e40a9c53f" containerName="payload-processing"
Sep 07 11:41:18.696542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696517    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" containerName="payload-pre-processing"
Sep 07 11:41:18.696542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696523    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" containerName="payload-pre-processing"
Sep 07 11:41:18.696542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696537    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" containerName="payload-processing"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696544    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" containerName="payload-processing"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696553    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696559    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696565    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="580a19d9-0450-4a37-a615-93fb8014fb81" containerName="revoke-keys"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696570    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="580a19d9-0450-4a37-a615-93fb8014fb81" containerName="revoke-keys"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696628    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3c50dc-86ea-4aa2-a0c5-4f94433a7a28" containerName="payload-processing"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696638    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" containerName="payload-pre-processing"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696643    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="108ad278-6cee-4e02-abd4-609e40a9c53f" containerName="payload-processing"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696651    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="580a19d9-0450-4a37-a615-93fb8014fb81" containerName="revoke-keys"
Sep 07 11:41:18.696698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.696659    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5787665-50f7-40a0-b9b7-0cbdc44ccf58" containerName="istio-proxy"
Sep 07 11:41:18.699976 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.699947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.702071 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.702051    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-6b944b-openshift-default-dockercfg-h88tn\""
Sep 07 11:41:18.711304 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.711281    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"]
Sep 07 11:41:18.812538 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812446    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-socket\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812538 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-data\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812538 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-credential-socket\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhbp6\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-kube-api-access-vhbp6\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-certs\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-token\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-podinfo\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-envoy\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.812850 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.812816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13fd4eb3-74a9-448b-83bc-56c180f6355d-istiod-ca-cert\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.868986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.868935    2560 generic.go:356] "Generic (PLEG): container finished" podID="db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" containerID="5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55" exitCode=0
Sep 07 11:41:18.868986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.868978    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"
Sep 07 11:41:18.869230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.869014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh" event={"ID":"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776","Type":"ContainerDied","Data":"5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55"}
Sep 07 11:41:18.869230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.869063    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh" event={"ID":"db35e068-f1c4-4d2c-8bc7-c1f31eb1d776","Type":"ContainerDied","Data":"bf4c86df3f4d503b40ab09b3663fc622e1a17b5a31b060fb064fd65dc1ca15ed"}
Sep 07 11:41:18.869230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.869086    2560 scope.go:117] "RemoveContainer" containerID="5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55"
Sep 07 11:41:18.870446 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.870423    2560 generic.go:356] "Generic (PLEG): container finished" podID="108ad278-6cee-4e02-abd4-609e40a9c53f" containerID="7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534" exitCode=0
Sep 07 11:41:18.870564 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.870473    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"
Sep 07 11:41:18.870564 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.870520    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5" event={"ID":"108ad278-6cee-4e02-abd4-609e40a9c53f","Type":"ContainerDied","Data":"7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534"}
Sep 07 11:41:18.870679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.870574    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5" event={"ID":"108ad278-6cee-4e02-abd4-609e40a9c53f","Type":"ContainerDied","Data":"3d759f1486dc701f2c41e3b767fc471ea68760402fdcbf17b536d4abf9eb65d2"}
Sep 07 11:41:18.878339 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.878321    2560 scope.go:117] "RemoveContainer" containerID="5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55"
Sep 07 11:41:18.878621 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:41:18.878601    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55\": container with ID starting with 5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55 not found: ID does not exist" containerID="5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55"
Sep 07 11:41:18.878674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.878630    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55"} err="failed to get container status \"5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55\": rpc error: code = NotFound desc = could not find container \"5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55\": container with ID starting with 5eb387024926013984d200876a6ac4d3c4ceefb11197c1bac3e2d2331873be55 not found: ID does not exist"
Sep 07 11:41:18.878674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.878645    2560 scope.go:117] "RemoveContainer" containerID="7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534"
Sep 07 11:41:18.886722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.886693    2560 scope.go:117] "RemoveContainer" containerID="7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534"
Sep 07 11:41:18.887054 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:41:18.887033    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534\": container with ID starting with 7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534 not found: ID does not exist" containerID="7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534"
Sep 07 11:41:18.887142 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.887059    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534"} err="failed to get container status \"7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534\": rpc error: code = NotFound desc = could not find container \"7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534\": container with ID starting with 7ead2a353e1e69d9f99c562f123598a7e3a96b6c9bc85eb422c5e4a614a5c534 not found: ID does not exist"
Sep 07 11:41:18.887516 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.887497    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"]
Sep 07 11:41:18.891566 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.891543    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d6b326a7-74d55fd4b9-9sxq5"]
Sep 07 11:41:18.901703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.901678    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"]
Sep 07 11:41:18.905798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.905776    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d6b326a7-6c875d5f78-xrkdh"]
Sep 07 11:41:18.914179 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914152    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-certs\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914256 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914197    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-token\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914325 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-podinfo\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914371 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-envoy\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914501 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914483    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-certs\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914580 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13fd4eb3-74a9-448b-83bc-56c180f6355d-istiod-ca-cert\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914639 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-socket\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914639 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-data\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914774 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-credential-socket\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914774 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914674    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhbp6\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-kube-api-access-vhbp6\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-socket\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.914979 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.914957    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-data\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.915190 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.915166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13fd4eb3-74a9-448b-83bc-56c180f6355d-istiod-ca-cert\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.915309 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.915290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-credential-socket\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.916453 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.916432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-envoy\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.916675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.916659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-podinfo\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.922364 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.922338    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-token\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:18.922515 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:18.922497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhbp6\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-kube-api-access-vhbp6\") pod \"e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") " pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:19.012385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.012351    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:19.287766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.287718    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"]
Sep 07 11:41:19.291143 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:41:19.291112    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fd4eb3_74a9_448b_83bc_56c180f6355d.slice/crio-887a3fc0cd76b85da5269c8777a3becb2eb579070a8781a6c05d476b072ebdef WatchSource:0}: Error finding container 887a3fc0cd76b85da5269c8777a3becb2eb579070a8781a6c05d476b072ebdef: Status 404 returned error can't find the container with id 887a3fc0cd76b85da5269c8777a3becb2eb579070a8781a6c05d476b072ebdef
Sep 07 11:41:19.293475 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.293441    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:41:19.293549 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.293512    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:41:19.293549 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.293539    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:41:19.877892 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.877858    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" event={"ID":"13fd4eb3-74a9-448b-83bc-56c180f6355d","Type":"ContainerStarted","Data":"1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"}
Sep 07 11:41:19.878113 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.877914    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" event={"ID":"13fd4eb3-74a9-448b-83bc-56c180f6355d","Type":"ContainerStarted","Data":"887a3fc0cd76b85da5269c8777a3becb2eb579070a8781a6c05d476b072ebdef"}
Sep 07 11:41:19.898923 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:19.898869    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podStartSLOduration=1.8988555059999999 podStartE2EDuration="1.898855506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:41:19.897430482 +0000 UTC m=+1111.844781266" watchObservedRunningTime="2026-09-07 11:41:19.898855506 +0000 UTC m=+1111.846206289"
Sep 07 11:41:20.012947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:20.012912    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:20.018151 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:20.018128    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:20.499094 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:20.499048    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="108ad278-6cee-4e02-abd4-609e40a9c53f" path="/var/lib/kubelet/pods/108ad278-6cee-4e02-abd4-609e40a9c53f/volumes"
Sep 07 11:41:20.499459 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:20.499445    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db35e068-f1c4-4d2c-8bc7-c1f31eb1d776" path="/var/lib/kubelet/pods/db35e068-f1c4-4d2c-8bc7-c1f31eb1d776/volumes"
Sep 07 11:41:20.885087 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:20.884999    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:20.886088 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:20.886067    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:23.733158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.733122    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"]
Sep 07 11:41:23.737935 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.737916    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.740551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.740532    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-6b944b-dockercfg-qpplm\""
Sep 07 11:41:23.740824 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.740808    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-6b944b\""
Sep 07 11:41:23.751493 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.751470    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"]
Sep 07 11:41:23.784855 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.784826    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"]
Sep 07 11:41:23.788268 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.788253    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:23.799959 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.799935    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"]
Sep 07 11:41:23.864926 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.864882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:23.865109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.864934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8c689f-1d27-4f58-8466-f6363bde92ac-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.865109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.865005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqwk\" (UniqueName: \"kubernetes.io/projected/ed8c689f-1d27-4f58-8466-f6363bde92ac-kube-api-access-4fqwk\") pod \"payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.865109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.865045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdrcc\" (UniqueName: \"kubernetes.io/projected/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-kube-api-access-mdrcc\") pod \"payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:23.965653 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.965619    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdrcc\" (UniqueName: \"kubernetes.io/projected/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-kube-api-access-mdrcc\") pod \"payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:23.965873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.965679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:23.965873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.965705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8c689f-1d27-4f58-8466-f6363bde92ac-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.965873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.965757    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqwk\" (UniqueName: \"kubernetes.io/projected/ed8c689f-1d27-4f58-8466-f6363bde92ac-kube-api-access-4fqwk\") pod \"payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.966440 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.966414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8c689f-1d27-4f58-8466-f6363bde92ac-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.966532 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.966443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:23.972731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.972708    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqwk\" (UniqueName: \"kubernetes.io/projected/ed8c689f-1d27-4f58-8466-f6363bde92ac-kube-api-access-4fqwk\") pod \"payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:23.972907 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:23.972890    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdrcc\" (UniqueName: \"kubernetes.io/projected/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-kube-api-access-mdrcc\") pod \"payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:24.048684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.048626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:24.099101 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.099057    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:24.328765 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.328677    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"]
Sep 07 11:41:24.331436 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:41:24.331409    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8c689f_1d27_4f58_8466_f6363bde92ac.slice/crio-f3fb574f86351c8e3d1dd21a18df12200aa50079b5de237604e2124e6d6956f9 WatchSource:0}: Error finding container f3fb574f86351c8e3d1dd21a18df12200aa50079b5de237604e2124e6d6956f9: Status 404 returned error can't find the container with id f3fb574f86351c8e3d1dd21a18df12200aa50079b5de237604e2124e6d6956f9
Sep 07 11:41:24.396644 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.396614    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"]
Sep 07 11:41:24.404020 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:41:24.403986    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291646c7_4f9c_43ac_be2a_ca9ed15a37d1.slice/crio-39cbb1539e0677541a161c522c145ca167cd4563c6c85fa3eddc6105e538a03c WatchSource:0}: Error finding container 39cbb1539e0677541a161c522c145ca167cd4563c6c85fa3eddc6105e538a03c: Status 404 returned error can't find the container with id 39cbb1539e0677541a161c522c145ca167cd4563c6c85fa3eddc6105e538a03c
Sep 07 11:41:24.909572 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.909530    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5" event={"ID":"291646c7-4f9c-43ac-be2a-ca9ed15a37d1","Type":"ContainerStarted","Data":"84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd"}
Sep 07 11:41:24.910042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.909597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5" event={"ID":"291646c7-4f9c-43ac-be2a-ca9ed15a37d1","Type":"ContainerStarted","Data":"39cbb1539e0677541a161c522c145ca167cd4563c6c85fa3eddc6105e538a03c"}
Sep 07 11:41:24.910042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.909620    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:24.911123 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.911092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh" event={"ID":"ed8c689f-1d27-4f58-8466-f6363bde92ac","Type":"ContainerStarted","Data":"9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae"}
Sep 07 11:41:24.911249 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.911143    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh" event={"ID":"ed8c689f-1d27-4f58-8466-f6363bde92ac","Type":"ContainerStarted","Data":"f3fb574f86351c8e3d1dd21a18df12200aa50079b5de237604e2124e6d6956f9"}
Sep 07 11:41:24.911249 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.911170    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:24.927773 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.927701    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5" podStartSLOduration=1.927682628 podStartE2EDuration="1.927682628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:41:24.926092834 +0000 UTC m=+1116.873443620" watchObservedRunningTime="2026-09-07 11:41:24.927682628 +0000 UTC m=+1116.875033417"
Sep 07 11:41:24.943057 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:24.943014    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh" podStartSLOduration=1.943000641 podStartE2EDuration="1.943000641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:41:24.941983602 +0000 UTC m=+1116.889334386" watchObservedRunningTime="2026-09-07 11:41:24.943000641 +0000 UTC m=+1116.890351425"
Sep 07 11:41:35.886605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.886526    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:35.886605 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.886592    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:35.920464 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.920436    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:41:35.920812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.920794    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:41:35.983880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.983816    2560 generic.go:356] "Generic (PLEG): container finished" podID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerID="1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b" exitCode=0
Sep 07 11:41:35.984035 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.983894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" event={"ID":"13fd4eb3-74a9-448b-83bc-56c180f6355d","Type":"ContainerDied","Data":"1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"}
Sep 07 11:41:35.984349 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.984333    2560 scope.go:117] "RemoveContainer" containerID="1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"
Sep 07 11:41:35.985623 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.985586    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:41:35.985707 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.985684    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:41:35.985778 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:35.985759    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:41:36.993320 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:36.993274    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" event={"ID":"13fd4eb3-74a9-448b-83bc-56c180f6355d","Type":"ContainerStarted","Data":"f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c"}
Sep 07 11:41:37.013110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:37.013073    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:37.013681 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:37.013486    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:37.015300 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:37.015256    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:38.013267 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:38.013227    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:38.013795 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:38.013290    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:39.013705 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:39.013657    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:39.014185 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:39.013764    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:40.013333 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:40.013294    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:40.013643 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:40.013613    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:41.013519 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:41.013477    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:41.014010 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:41.013823    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:42.013201 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:42.013156    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 11:41:42.013409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:42.013229    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 11:41:43.018309 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:43.018278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:43.018758 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:43.018565    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:43.019408 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:43.019386    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:41:46.273592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.273561    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"]
Sep 07 11:41:46.276414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.276396    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.278815 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.278782    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-0c816d\""
Sep 07 11:41:46.278923 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.278824    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-0c816d-dockercfg-ncmpw\""
Sep 07 11:41:46.286799 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.286776    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"]
Sep 07 11:41:46.387638 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.387595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e848345-8956-476f-9ee7-8e0ff6a4d692-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.387829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.387655    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8c9h\" (UniqueName: \"kubernetes.io/projected/1e848345-8956-476f-9ee7-8e0ff6a4d692-kube-api-access-x8c9h\") pod \"payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.488910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.488871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e848345-8956-476f-9ee7-8e0ff6a4d692-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.489080 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.488924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8c9h\" (UniqueName: \"kubernetes.io/projected/1e848345-8956-476f-9ee7-8e0ff6a4d692-kube-api-access-x8c9h\") pod \"payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.489610 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.489587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e848345-8956-476f-9ee7-8e0ff6a4d692-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.497438 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.497405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8c9h\" (UniqueName: \"kubernetes.io/projected/1e848345-8956-476f-9ee7-8e0ff6a4d692-kube-api-access-x8c9h\") pod \"payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.587569 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.587488    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:46.860506 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:46.860479    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"]
Sep 07 11:41:46.862724 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:41:46.862699    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e848345_8956_476f_9ee7_8e0ff6a4d692.slice/crio-1c93d037554c69b0b87bd9f08515c9550dbcb9693e44b41d611dbfbaeea9c4a6 WatchSource:0}: Error finding container 1c93d037554c69b0b87bd9f08515c9550dbcb9693e44b41d611dbfbaeea9c4a6: Status 404 returned error can't find the container with id 1c93d037554c69b0b87bd9f08515c9550dbcb9693e44b41d611dbfbaeea9c4a6
Sep 07 11:41:47.056869 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:47.056834    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj" event={"ID":"1e848345-8956-476f-9ee7-8e0ff6a4d692","Type":"ContainerStarted","Data":"c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107"}
Sep 07 11:41:47.057037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:47.056895    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj" event={"ID":"1e848345-8956-476f-9ee7-8e0ff6a4d692","Type":"ContainerStarted","Data":"1c93d037554c69b0b87bd9f08515c9550dbcb9693e44b41d611dbfbaeea9c4a6"}
Sep 07 11:41:47.057037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:47.056940    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:41:47.076469 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:47.076422    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj" podStartSLOduration=1.076404955 podStartE2EDuration="1.076404955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:41:47.075014179 +0000 UTC m=+1139.022364964" watchObservedRunningTime="2026-09-07 11:41:47.076404955 +0000 UTC m=+1139.023755779"
Sep 07 11:41:52.033620 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.033586    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"]
Sep 07 11:41:52.037576 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.037553    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.039612 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.039588    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-0f0a2f\""
Sep 07 11:41:52.039720 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.039704    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-0f0a2f-dockercfg-tg9sm\""
Sep 07 11:41:52.046801 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.046778    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"]
Sep 07 11:41:52.138961 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.138929    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gr5q\" (UniqueName: \"kubernetes.io/projected/628ef7d0-f696-4773-9a78-670ee59d2cbd-kube-api-access-7gr5q\") pod \"payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.139177 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.138983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628ef7d0-f696-4773-9a78-670ee59d2cbd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.240309 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.240268    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gr5q\" (UniqueName: \"kubernetes.io/projected/628ef7d0-f696-4773-9a78-670ee59d2cbd-kube-api-access-7gr5q\") pod \"payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.240476 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.240339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628ef7d0-f696-4773-9a78-670ee59d2cbd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.241063 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.241044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628ef7d0-f696-4773-9a78-670ee59d2cbd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.252189 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.252167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gr5q\" (UniqueName: \"kubernetes.io/projected/628ef7d0-f696-4773-9a78-670ee59d2cbd-kube-api-access-7gr5q\") pod \"payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.349023 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.348933    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:52.622724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:52.622699    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"]
Sep 07 11:41:52.625536 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:41:52.625505    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628ef7d0_f696_4773_9a78_670ee59d2cbd.slice/crio-daa8b339b185e89fbc0f71cca54a3f80fefd3947980c1a8dbbc3499659364783 WatchSource:0}: Error finding container daa8b339b185e89fbc0f71cca54a3f80fefd3947980c1a8dbbc3499659364783: Status 404 returned error can't find the container with id daa8b339b185e89fbc0f71cca54a3f80fefd3947980c1a8dbbc3499659364783
Sep 07 11:41:53.088600 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:53.088563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj" event={"ID":"628ef7d0-f696-4773-9a78-670ee59d2cbd","Type":"ContainerStarted","Data":"186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2"}
Sep 07 11:41:53.089047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:53.088617    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj" event={"ID":"628ef7d0-f696-4773-9a78-670ee59d2cbd","Type":"ContainerStarted","Data":"daa8b339b185e89fbc0f71cca54a3f80fefd3947980c1a8dbbc3499659364783"}
Sep 07 11:41:53.089047 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:53.088772    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:41:53.104977 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:53.104924    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj" podStartSLOduration=1.104909755 podStartE2EDuration="1.104909755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:41:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:41:53.104011678 +0000 UTC m=+1145.051362453" watchObservedRunningTime="2026-09-07 11:41:53.104909755 +0000 UTC m=+1145.052260539"
Sep 07 11:41:58.063928 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:41:58.063898    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:42:04.096395 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:04.096363    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:42:09.687344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.687306    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"]
Sep 07 11:42:09.691658 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.691632    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.693782 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.693730    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-2195e0-openshift-default-dockercfg-hfkwr\""
Sep 07 11:42:09.705414 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.703495    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"]
Sep 07 11:42:09.791629 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791588    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-credential-socket\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.791629 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791635    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-token\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.791865 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-data\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.791865 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5k2n\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-kube-api-access-z5k2n\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.791865 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istiod-ca-cert\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.792005 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791870    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-socket\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.792005 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.791967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-podinfo\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.792091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.792037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-envoy\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.792091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.792059    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-certs\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893537 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893499    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istiod-ca-cert\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893537 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893541    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-socket\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-podinfo\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-envoy\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-certs\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893649    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-credential-socket\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893674    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-token\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-data\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.893848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.893778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5k2n\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-kube-api-access-z5k2n\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.894172 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.894018    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-socket\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.894227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.894179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-credential-socket\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.894285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.894244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-certs\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.894342 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.894293    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-data\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.894516 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.894495    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istiod-ca-cert\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.896139 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.896110    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-envoy\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.896276 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.896256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-podinfo\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.901331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.901310    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5k2n\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-kube-api-access-z5k2n\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:09.901502 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:09.901485    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-token\") pod \"e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") " pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:10.006783 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:10.006729    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:10.283213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:10.283188    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"]
Sep 07 11:42:10.285727 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:42:10.285698    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ca5647_73aa_4ec7_ba83_8ccd64805ae0.slice/crio-306c24655c4303aff0a6371c93d1e5c89339b997182fb28b5d18af037f4d36c2 WatchSource:0}: Error finding container 306c24655c4303aff0a6371c93d1e5c89339b997182fb28b5d18af037f4d36c2: Status 404 returned error can't find the container with id 306c24655c4303aff0a6371c93d1e5c89339b997182fb28b5d18af037f4d36c2
Sep 07 11:42:10.288120 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:10.288084    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:10.288217 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:10.288165    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:10.288217 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:10.288209    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:11.186648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:11.186569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" event={"ID":"41ca5647-73aa-4ec7-ba83-8ccd64805ae0","Type":"ContainerStarted","Data":"12f529cbb294c9a2a7d02f0c89b21e8468593f9e162e087dd54656786be6870e"}
Sep 07 11:42:11.186648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:11.186627    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" event={"ID":"41ca5647-73aa-4ec7-ba83-8ccd64805ae0","Type":"ContainerStarted","Data":"306c24655c4303aff0a6371c93d1e5c89339b997182fb28b5d18af037f4d36c2"}
Sep 07 11:42:11.208600 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:11.208548    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podStartSLOduration=2.208532854 podStartE2EDuration="2.208532854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:42:11.206034454 +0000 UTC m=+1163.153385238" watchObservedRunningTime="2026-09-07 11:42:11.208532854 +0000 UTC m=+1163.155883697"
Sep 07 11:42:12.007597 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:12.007555    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:12.013272 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:12.013244    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:12.191820 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:12.191785    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:12.192725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:12.192707    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:15.159933 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.159899    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"]
Sep 07 11:42:15.162847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.162831    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.165179 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.165157    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-2195e0-dockercfg-txr9n\""
Sep 07 11:42:15.165289 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.165199    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-2195e0\""
Sep 07 11:42:15.174097 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.174073    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"]
Sep 07 11:42:15.248479 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.248443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ctl\" (UniqueName: \"kubernetes.io/projected/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-kube-api-access-p9ctl\") pod \"payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.248799 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.248736    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.349987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.349947    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ctl\" (UniqueName: \"kubernetes.io/projected/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-kube-api-access-p9ctl\") pod \"payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.350187 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.350003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.350705 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.350679    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.358105 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.358072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ctl\" (UniqueName: \"kubernetes.io/projected/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-kube-api-access-p9ctl\") pod \"payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.474958 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.474876    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:15.743418 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:15.743391    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"]
Sep 07 11:42:15.745563 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:42:15.745535    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39db941a_fcc2_4b0c_a034_98e1e84b1bf9.slice/crio-c9ccfd6dbb5f7c1bda1cd24978fd007bc0fe051f0c723906d9d6b2747f41890d WatchSource:0}: Error finding container c9ccfd6dbb5f7c1bda1cd24978fd007bc0fe051f0c723906d9d6b2747f41890d: Status 404 returned error can't find the container with id c9ccfd6dbb5f7c1bda1cd24978fd007bc0fe051f0c723906d9d6b2747f41890d
Sep 07 11:42:16.215191 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:16.215097    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx" event={"ID":"39db941a-fcc2-4b0c-a034-98e1e84b1bf9","Type":"ContainerStarted","Data":"c8c3e00ea43e36c6ea3809574bdaf535ccca06629a3c279af06a9f86ee1a6505"}
Sep 07 11:42:16.215191 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:16.215147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx" event={"ID":"39db941a-fcc2-4b0c-a034-98e1e84b1bf9","Type":"ContainerStarted","Data":"c9ccfd6dbb5f7c1bda1cd24978fd007bc0fe051f0c723906d9d6b2747f41890d"}
Sep 07 11:42:16.215610 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:16.215260    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:16.231356 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:16.231311    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx" podStartSLOduration=1.231292742 podStartE2EDuration="1.231292742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:42:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:42:16.230642949 +0000 UTC m=+1168.177993737" watchObservedRunningTime="2026-09-07 11:42:16.231292742 +0000 UTC m=+1168.178643527"
Sep 07 11:42:27.222493 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:27.222450    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:42:31.298867 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:31.298828    2560 generic.go:356] "Generic (PLEG): container finished" podID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerID="12f529cbb294c9a2a7d02f0c89b21e8468593f9e162e087dd54656786be6870e" exitCode=0
Sep 07 11:42:31.299292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:31.298898    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" event={"ID":"41ca5647-73aa-4ec7-ba83-8ccd64805ae0","Type":"ContainerDied","Data":"12f529cbb294c9a2a7d02f0c89b21e8468593f9e162e087dd54656786be6870e"}
Sep 07 11:42:31.299292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:31.299268    2560 scope.go:117] "RemoveContainer" containerID="12f529cbb294c9a2a7d02f0c89b21e8468593f9e162e087dd54656786be6870e"
Sep 07 11:42:31.300360 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:31.300331    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:31.300422 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:31.300411    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:31.300463 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:31.300453    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:32.309986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:32.309940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" event={"ID":"41ca5647-73aa-4ec7-ba83-8ccd64805ae0","Type":"ContainerStarted","Data":"a3eb13117c263d95fefa2ad09b62aae77106a0d1374162e1ee0f8a05aa7a8311"}
Sep 07 11:42:33.007632 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:33.007588    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:33.007938 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:33.007663    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 07 11:42:33.008157 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:33.008116    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 07 11:42:34.007827 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:34.007787    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 07 11:42:34.008319 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:34.007854    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 07 11:42:35.007395 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:35.007350    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 07 11:42:35.007593 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:35.007419    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 07 11:42:36.007700 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:36.007651    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 07 11:42:36.008228 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:36.007728    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 07 11:42:37.007415 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:37.007377    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 07 11:42:37.007724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:37.007688    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 07 11:42:38.011299 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:38.011268    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:38.011731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:38.011545    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:38.012357 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:38.012343    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:42:48.564114 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:48.564079    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:42:48.566453 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:48.566423    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:42:51.726790 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.726732    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"]
Sep 07 11:42:51.729853 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.729822    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.733050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.733026    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8108f8fd-openshift-default-dockercfg-mv4bc\""
Sep 07 11:42:51.745858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.745838    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"]
Sep 07 11:42:51.799829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.799799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-podinfo\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.799829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.799833    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-token\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.799878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-certs\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.799911    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-socket\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.799932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-credential-socket\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.799967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-envoy\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.800007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80066d-23b1-49a3-9f6e-18da653ef59c-istiod-ca-cert\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.800023    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmf8d\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-kube-api-access-kmf8d\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.800244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.800094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-data\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.900994 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.900948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-podinfo\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.900994 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901000    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-token\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-certs\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901090    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-socket\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-credential-socket\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-envoy\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80066d-23b1-49a3-9f6e-18da653ef59c-istiod-ca-cert\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901271 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmf8d\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-kube-api-access-kmf8d\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-data\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-certs\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901590    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-socket\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-credential-socket\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901778    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-data\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.901993 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.901968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80066d-23b1-49a3-9f6e-18da653ef59c-istiod-ca-cert\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.903526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.903489    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-envoy\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.904065 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.904045    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-podinfo\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.909009 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.908986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-token\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:51.909422 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:51.909405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmf8d\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-kube-api-access-kmf8d\") pod \"e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") " pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:52.040205 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.040107    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:52.318244 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.318221    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"]
Sep 07 11:42:52.320244 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:42:52.320213    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e80066d_23b1_49a3_9f6e_18da653ef59c.slice/crio-609058f88f9f76ede435055c177621e736b28945f0e6cefbb325b03c88990115 WatchSource:0}: Error finding container 609058f88f9f76ede435055c177621e736b28945f0e6cefbb325b03c88990115: Status 404 returned error can't find the container with id 609058f88f9f76ede435055c177621e736b28945f0e6cefbb325b03c88990115
Sep 07 11:42:52.322730 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.322699    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:52.322805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.322787    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:52.322846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.322819    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:42:52.416123 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.416083    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" event={"ID":"0e80066d-23b1-49a3-9f6e-18da653ef59c","Type":"ContainerStarted","Data":"3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"}
Sep 07 11:42:52.416289 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.416158    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" event={"ID":"0e80066d-23b1-49a3-9f6e-18da653ef59c","Type":"ContainerStarted","Data":"609058f88f9f76ede435055c177621e736b28945f0e6cefbb325b03c88990115"}
Sep 07 11:42:52.436724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:52.436652    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podStartSLOduration=1.436636117 podStartE2EDuration="1.436636117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:42:52.434779983 +0000 UTC m=+1204.382130768" watchObservedRunningTime="2026-09-07 11:42:52.436636117 +0000 UTC m=+1204.383986901"
Sep 07 11:42:53.040810 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:53.040777    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:53.046081 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:53.046056    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:53.421985 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:53.421896    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:53.422907 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:53.422888    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:42:56.441587 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.441555    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"]
Sep 07 11:42:56.444290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.444272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.447685 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.447659    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8108f8fd-dockercfg-wq2xq\""
Sep 07 11:42:56.447813 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.447683    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8108f8fd\""
Sep 07 11:42:56.461725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.461695    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"]
Sep 07 11:42:56.545071 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.545035    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2nbl\" (UniqueName: \"kubernetes.io/projected/16b2d958-f278-4880-aeb8-1ba5707c158e-kube-api-access-q2nbl\") pod \"payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.545242 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.545158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16b2d958-f278-4880-aeb8-1ba5707c158e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.646103 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.646071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2nbl\" (UniqueName: \"kubernetes.io/projected/16b2d958-f278-4880-aeb8-1ba5707c158e-kube-api-access-q2nbl\") pod \"payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.646262 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.646135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16b2d958-f278-4880-aeb8-1ba5707c158e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.646839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.646815    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16b2d958-f278-4880-aeb8-1ba5707c158e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.654766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.654729    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2nbl\" (UniqueName: \"kubernetes.io/projected/16b2d958-f278-4880-aeb8-1ba5707c158e-kube-api-access-q2nbl\") pod \"payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:56.755054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:56.755027    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:57.030157 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:57.030134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"]
Sep 07 11:42:57.031895 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:42:57.031864    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b2d958_f278_4880_aeb8_1ba5707c158e.slice/crio-7ceea80bd41637575de4292eab20bceb7e91411747318a8a50ae812a097972c2 WatchSource:0}: Error finding container 7ceea80bd41637575de4292eab20bceb7e91411747318a8a50ae812a097972c2: Status 404 returned error can't find the container with id 7ceea80bd41637575de4292eab20bceb7e91411747318a8a50ae812a097972c2
Sep 07 11:42:57.445671 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:57.445579    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw" event={"ID":"16b2d958-f278-4880-aeb8-1ba5707c158e","Type":"ContainerStarted","Data":"58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a"}
Sep 07 11:42:57.445671 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:57.445629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw" event={"ID":"16b2d958-f278-4880-aeb8-1ba5707c158e","Type":"ContainerStarted","Data":"7ceea80bd41637575de4292eab20bceb7e91411747318a8a50ae812a097972c2"}
Sep 07 11:42:57.445671 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:57.445661    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:42:57.469731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:42:57.469682    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw" podStartSLOduration=1.469668729 podStartE2EDuration="1.469668729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:42:57.467319061 +0000 UTC m=+1209.414669846" watchObservedRunningTime="2026-09-07 11:42:57.469668729 +0000 UTC m=+1209.417019513"
Sep 07 11:43:08.458659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:08.458625    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:43:16.568968 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:16.568895    2560 generic.go:356] "Generic (PLEG): container finished" podID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerID="3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99" exitCode=0
Sep 07 11:43:16.569331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:16.568967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" event={"ID":"0e80066d-23b1-49a3-9f6e-18da653ef59c","Type":"ContainerDied","Data":"3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"}
Sep 07 11:43:16.569331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:16.569320    2560 scope.go:117] "RemoveContainer" containerID="3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"
Sep 07 11:43:16.570385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:16.570355    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:43:16.570440 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:16.570422    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:43:16.570479 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:16.570456    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:43:17.576323 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:17.576282    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" event={"ID":"0e80066d-23b1-49a3-9f6e-18da653ef59c","Type":"ContainerStarted","Data":"6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec"}
Sep 07 11:43:18.041040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:18.041001    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:43:18.041216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:18.041181    2560 patch_prober.go:28] interesting pod/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 07 11:43:18.041263 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:18.041231    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 07 11:43:19.041321 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:19.041286    2560 patch_prober.go:28] interesting pod/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 07 11:43:19.041877 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:19.041350    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 07 11:43:20.041420 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:20.041381    2560 patch_prober.go:28] interesting pod/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 07 11:43:20.041897 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:20.041440    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 07 11:43:21.041018 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:21.040979    2560 patch_prober.go:28] interesting pod/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 07 11:43:21.041205 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:21.041054    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 07 11:43:22.040846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:22.040807    2560 patch_prober.go:28] interesting pod/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 07 11:43:22.041331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:22.040865    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 07 11:43:23.045197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:23.045167    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:43:23.045646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:23.045564    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:43:23.046342 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:43:23.046322    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:44:40.545792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.545732    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"]
Sep 07 11:44:40.546358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.546017    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw" podUID="16b2d958-f278-4880-aeb8-1ba5707c158e" containerName="payload-pre-processing" containerID="cri-o://58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a" gracePeriod=30
Sep 07 11:44:40.836089 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.836055    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:44:40.900555 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.900518    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2nbl\" (UniqueName: \"kubernetes.io/projected/16b2d958-f278-4880-aeb8-1ba5707c158e-kube-api-access-q2nbl\") pod \"16b2d958-f278-4880-aeb8-1ba5707c158e\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") "
Sep 07 11:44:40.900770 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.900642    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16b2d958-f278-4880-aeb8-1ba5707c158e-plugins-config-volume\") pod \"16b2d958-f278-4880-aeb8-1ba5707c158e\" (UID: \"16b2d958-f278-4880-aeb8-1ba5707c158e\") "
Sep 07 11:44:40.901032 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.901008    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16b2d958-f278-4880-aeb8-1ba5707c158e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16b2d958-f278-4880-aeb8-1ba5707c158e" (UID: "16b2d958-f278-4880-aeb8-1ba5707c158e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:44:40.902619 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:40.902598    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b2d958-f278-4880-aeb8-1ba5707c158e-kube-api-access-q2nbl" (OuterVolumeSpecName: "kube-api-access-q2nbl") pod "16b2d958-f278-4880-aeb8-1ba5707c158e" (UID: "16b2d958-f278-4880-aeb8-1ba5707c158e"). InnerVolumeSpecName "kube-api-access-q2nbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:44:41.002040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.002002    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2nbl\" (UniqueName: \"kubernetes.io/projected/16b2d958-f278-4880-aeb8-1ba5707c158e-kube-api-access-q2nbl\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:44:41.002040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.002033    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16b2d958-f278-4880-aeb8-1ba5707c158e-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:44:41.047706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.047675    2560 generic.go:356] "Generic (PLEG): container finished" podID="16b2d958-f278-4880-aeb8-1ba5707c158e" containerID="58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a" exitCode=0
Sep 07 11:44:41.047910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.047717    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"
Sep 07 11:44:41.047910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.047772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw" event={"ID":"16b2d958-f278-4880-aeb8-1ba5707c158e","Type":"ContainerDied","Data":"58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a"}
Sep 07 11:44:41.047910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.047831    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw" event={"ID":"16b2d958-f278-4880-aeb8-1ba5707c158e","Type":"ContainerDied","Data":"7ceea80bd41637575de4292eab20bceb7e91411747318a8a50ae812a097972c2"}
Sep 07 11:44:41.047910 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.047860    2560 scope.go:117] "RemoveContainer" containerID="58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a"
Sep 07 11:44:41.057818 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.057801    2560 scope.go:117] "RemoveContainer" containerID="58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a"
Sep 07 11:44:41.058110 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:44:41.058093    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a\": container with ID starting with 58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a not found: ID does not exist" containerID="58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a"
Sep 07 11:44:41.058166 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.058115    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a"} err="failed to get container status \"58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a\": rpc error: code = NotFound desc = could not find container \"58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a\": container with ID starting with 58d957e7a299e4aba45d887e30e31971700fc9d94cee6ad8704fe8c39437b21a not found: ID does not exist"
Sep 07 11:44:41.070550 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.070517    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"]
Sep 07 11:44:41.072523 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:41.072504    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8108f8fd-dcc4f6b46-mqtsw"]
Sep 07 11:44:42.500208 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:42.500172    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16b2d958-f278-4880-aeb8-1ba5707c158e" path="/var/lib/kubelet/pods/16b2d958-f278-4880-aeb8-1ba5707c158e/volumes"
Sep 07 11:44:48.676280 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:44:48.676248    2560 scope.go:117] "RemoveContainer" containerID="66cd139cc494df04f5241a4870a20b3d83637868e1f879152bd49b4873a63880"
Sep 07 11:45:00.151158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.151123    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"]
Sep 07 11:45:00.151945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.151697    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b2d958-f278-4880-aeb8-1ba5707c158e" containerName="payload-pre-processing"
Sep 07 11:45:00.151945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.151717    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b2d958-f278-4880-aeb8-1ba5707c158e" containerName="payload-pre-processing"
Sep 07 11:45:00.151945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.151871    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b2d958-f278-4880-aeb8-1ba5707c158e" containerName="payload-pre-processing"
Sep 07 11:45:00.155216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.155196    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"]
Sep 07 11:45:00.155331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.155305    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"
Sep 07 11:45:00.157670 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.157643    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:45:00.157790 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.157710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"
Sep 07 11:45:00.167548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.167525    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"]
Sep 07 11:45:00.168987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.168854    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"
Sep 07 11:45:00.172927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.172437    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"]
Sep 07 11:45:00.180380 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.180360    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"
Sep 07 11:45:00.474198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.474172    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"]
Sep 07 11:45:00.475941 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:45:00.475908    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c4b82e_759d_439f_9103_902521b02d4d.slice/crio-f2f8c6b535a18f61bb0d92ed404e64a3e3d4652a21ea08cbfc71810f96116be5 WatchSource:0}: Error finding container f2f8c6b535a18f61bb0d92ed404e64a3e3d4652a21ea08cbfc71810f96116be5: Status 404 returned error can't find the container with id f2f8c6b535a18f61bb0d92ed404e64a3e3d4652a21ea08cbfc71810f96116be5
Sep 07 11:45:00.484407 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:00.484382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"]
Sep 07 11:45:01.163847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:01.163757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx" event={"ID":"2d7b65e8-d5c9-41f0-9571-0230b299ddb7","Type":"ContainerStarted","Data":"d499c79b5a7a2ed2a13a17199972646082288a5e61e2d02ba4a988fe171abe4f"}
Sep 07 11:45:01.163847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:01.163820    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx" event={"ID":"2d7b65e8-d5c9-41f0-9571-0230b299ddb7","Type":"ContainerStarted","Data":"90c3b482014b54311e8944e3969ffd90d86877182a1d07bbfc8684c3e2f8d5d2"}
Sep 07 11:45:01.165350 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:01.165326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7" event={"ID":"24c4b82e-759d-439f-9103-902521b02d4d","Type":"ContainerStarted","Data":"72722c089c30cfab4fcec517f689f2cf707adf388a51411bd04c64d9d366672d"}
Sep 07 11:45:01.165465 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:01.165365    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7" event={"ID":"24c4b82e-759d-439f-9103-902521b02d4d","Type":"ContainerStarted","Data":"f2f8c6b535a18f61bb0d92ed404e64a3e3d4652a21ea08cbfc71810f96116be5"}
Sep 07 11:45:01.180957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:01.180914    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx" podStartSLOduration=1.180899864 podStartE2EDuration="1.180899864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:45:01.179551725 +0000 UTC m=+1333.126902507" watchObservedRunningTime="2026-09-07 11:45:01.180899864 +0000 UTC m=+1333.128250638"
Sep 07 11:45:01.202608 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:01.202567    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7" podStartSLOduration=1.2025540239999999 podStartE2EDuration="1.202554024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:45:01.200973027 +0000 UTC m=+1333.148323811" watchObservedRunningTime="2026-09-07 11:45:01.202554024 +0000 UTC m=+1333.149904807"
Sep 07 11:45:02.173542 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:02.173503    2560 generic.go:356] "Generic (PLEG): container finished" podID="2d7b65e8-d5c9-41f0-9571-0230b299ddb7" containerID="d499c79b5a7a2ed2a13a17199972646082288a5e61e2d02ba4a988fe171abe4f" exitCode=0
Sep 07 11:45:02.173987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:02.173579    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx" event={"ID":"2d7b65e8-d5c9-41f0-9571-0230b299ddb7","Type":"ContainerDied","Data":"d499c79b5a7a2ed2a13a17199972646082288a5e61e2d02ba4a988fe171abe4f"}
Sep 07 11:45:02.175264 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:02.175240    2560 generic.go:356] "Generic (PLEG): container finished" podID="24c4b82e-759d-439f-9103-902521b02d4d" containerID="72722c089c30cfab4fcec517f689f2cf707adf388a51411bd04c64d9d366672d" exitCode=0
Sep 07 11:45:02.175390 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:02.175272    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7" event={"ID":"24c4b82e-759d-439f-9103-902521b02d4d","Type":"ContainerDied","Data":"72722c089c30cfab4fcec517f689f2cf707adf388a51411bd04c64d9d366672d"}
Sep 07 11:45:03.310642 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:03.310617    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"
Sep 07 11:45:03.350689 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:03.350667    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"
Sep 07 11:45:04.187126 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:04.187096    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"
Sep 07 11:45:04.187126 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:04.187112    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx" event={"ID":"2d7b65e8-d5c9-41f0-9571-0230b299ddb7","Type":"ContainerDied","Data":"90c3b482014b54311e8944e3969ffd90d86877182a1d07bbfc8684c3e2f8d5d2"}
Sep 07 11:45:04.187358 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:04.187166    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90c3b482014b54311e8944e3969ffd90d86877182a1d07bbfc8684c3e2f8d5d2"
Sep 07 11:45:04.188431 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:04.188411    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"
Sep 07 11:45:04.188570 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:04.188451    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7" event={"ID":"24c4b82e-759d-439f-9103-902521b02d4d","Type":"ContainerDied","Data":"f2f8c6b535a18f61bb0d92ed404e64a3e3d4652a21ea08cbfc71810f96116be5"}
Sep 07 11:45:04.188570 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:04.188491    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2f8c6b535a18f61bb0d92ed404e64a3e3d4652a21ea08cbfc71810f96116be5"
Sep 07 11:45:33.330828 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:33.330793    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"]
Sep 07 11:45:33.331379 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:33.331059    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" containerID="cri-o://6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec" gracePeriod=30
Sep 07 11:45:38.048003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.047962    2560 patch_prober.go:28] interesting pod/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 11:45:38.048368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.048031    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 11:45:38.586487 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.586465    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:45:38.748573 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748542    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80066d-23b1-49a3-9f6e-18da653ef59c-istiod-ca-cert\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748573 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748575    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-envoy\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748596    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-certs\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748636    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-token\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748653    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmf8d\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-kube-api-access-kmf8d\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748677    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-credential-socket\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748732    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-socket\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.748858 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748781    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-data\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.749173 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.748863    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-podinfo\") pod \"0e80066d-23b1-49a3-9f6e-18da653ef59c\" (UID: \"0e80066d-23b1-49a3-9f6e-18da653ef59c\") "
Sep 07 11:45:38.749173 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749039    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:45:38.749173 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749044    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e80066d-23b1-49a3-9f6e-18da653ef59c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:45:38.749344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749199    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:45:38.749344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749258    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:45:38.749344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749268    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e80066d-23b1-49a3-9f6e-18da653ef59c-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.749344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749287    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.749548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.749351    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:45:38.750923 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.750892    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:45:38.751040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.750983    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:45:38.751040 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.750979    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-kube-api-access-kmf8d" (OuterVolumeSpecName: "kube-api-access-kmf8d") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "kube-api-access-kmf8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:45:38.751136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.751043    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e80066d-23b1-49a3-9f6e-18da653ef59c" (UID: "0e80066d-23b1-49a3-9f6e-18da653ef59c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:45:38.849723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849699    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.849723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849719    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.849723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849729    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.849954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849758    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.849954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849769    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmf8d\" (UniqueName: \"kubernetes.io/projected/0e80066d-23b1-49a3-9f6e-18da653ef59c-kube-api-access-kmf8d\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.849954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849779    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:38.849954 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:38.849787    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e80066d-23b1-49a3-9f6e-18da653ef59c-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:45:39.376830 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.376793    2560 generic.go:356] "Generic (PLEG): container finished" podID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerID="6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec" exitCode=0
Sep 07 11:45:39.377223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.376849    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"
Sep 07 11:45:39.377223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.376882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" event={"ID":"0e80066d-23b1-49a3-9f6e-18da653ef59c","Type":"ContainerDied","Data":"6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec"}
Sep 07 11:45:39.377223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.376946    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc" event={"ID":"0e80066d-23b1-49a3-9f6e-18da653ef59c","Type":"ContainerDied","Data":"609058f88f9f76ede435055c177621e736b28945f0e6cefbb325b03c88990115"}
Sep 07 11:45:39.377223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.376970    2560 scope.go:117] "RemoveContainer" containerID="6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec"
Sep 07 11:45:39.386704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.386682    2560 scope.go:117] "RemoveContainer" containerID="3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"
Sep 07 11:45:39.400054 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.400003    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"]
Sep 07 11:45:39.401094 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.400681    2560 scope.go:117] "RemoveContainer" containerID="6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec"
Sep 07 11:45:39.401648 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:45:39.401506    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec\": container with ID starting with 6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec not found: ID does not exist" containerID="6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec"
Sep 07 11:45:39.401648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.401541    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec"} err="failed to get container status \"6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec\": rpc error: code = NotFound desc = could not find container \"6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec\": container with ID starting with 6b4d388eafc61c41d3762e042b3cf7760120b4a9c82ecd29c43f6febc85a43ec not found: ID does not exist"
Sep 07 11:45:39.401648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.401566    2560 scope.go:117] "RemoveContainer" containerID="3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"
Sep 07 11:45:39.402342 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:45:39.402284    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99\": container with ID starting with 3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99 not found: ID does not exist" containerID="3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"
Sep 07 11:45:39.402342 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.402319    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99"} err="failed to get container status \"3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99\": rpc error: code = NotFound desc = could not find container \"3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99\": container with ID starting with 3d0359bf21d7f40e78792bb688fc40db212e2cd9b506fbdfbe304349e25adc99 not found: ID does not exist"
Sep 07 11:45:39.403889 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:39.403868    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8108f8fd-openshift-default-74c546d98d-4lgnc"]
Sep 07 11:45:40.497446 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:45:40.497410    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" path="/var/lib/kubelet/pods/0e80066d-23b1-49a3-9f6e-18da653ef59c/volumes"
Sep 07 11:46:28.666089 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666056    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"]
Sep 07 11:46:28.666689 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666668    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666692    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666706    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d7b65e8-d5c9-41f0-9571-0230b299ddb7" containerName="cleanup"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666716    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d7b65e8-d5c9-41f0-9571-0230b299ddb7" containerName="cleanup"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666734    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666759    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666769    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c4b82e-759d-439f-9103-902521b02d4d" containerName="cleanup"
Sep 07 11:46:28.666797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666776    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c4b82e-759d-439f-9103-902521b02d4d" containerName="cleanup"
Sep 07 11:46:28.667155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666898    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c4b82e-759d-439f-9103-902521b02d4d" containerName="cleanup"
Sep 07 11:46:28.667155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666914    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d7b65e8-d5c9-41f0-9571-0230b299ddb7" containerName="cleanup"
Sep 07 11:46:28.667155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666928    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy"
Sep 07 11:46:28.667155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.666937    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e80066d-23b1-49a3-9f6e-18da653ef59c" containerName="istio-proxy"
Sep 07 11:46:28.670195 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.670173    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.672384 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.672360    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2cfab4-openshift-default-dockercfg-5tn2r\""
Sep 07 11:46:28.682299 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.682277    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"]
Sep 07 11:46:28.833659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833619    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-data\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833663    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-socket\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833689    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-envoy\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-certs\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-token\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833848    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-podinfo\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cxqp\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-kube-api-access-4cxqp\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.833912 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.833902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-credential-socket\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.834122 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.834046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8864574-66ca-4c7c-a641-221ae21d1bae-istiod-ca-cert\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935448 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8864574-66ca-4c7c-a641-221ae21d1bae-istiod-ca-cert\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935448 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-data\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935448 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-socket\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935448 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935438    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-envoy\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-certs\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-token\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-podinfo\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.935792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cxqp\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-kube-api-access-4cxqp\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.936003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935846    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-data\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.936003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-credential-socket\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.936003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.935926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-socket\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.936139 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.936071    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-certs\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.936231 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.936208    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8864574-66ca-4c7c-a641-221ae21d1bae-istiod-ca-cert\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.936288 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.936234    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-credential-socket\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.937701 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.937680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-envoy\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.938163 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.938141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-podinfo\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.944090 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.944071    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cxqp\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-kube-api-access-4cxqp\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.944376 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.944357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-token\") pod \"e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") " pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:28.982303 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:28.982277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:29.256309 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.256283    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"]
Sep 07 11:46:29.258264 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:46:29.258235    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8864574_66ca_4c7c_a641_221ae21d1bae.slice/crio-f94e00891e3e9647d3753812c92f9c2fa0882a2fd144be0f798a6c45b37e5b37 WatchSource:0}: Error finding container f94e00891e3e9647d3753812c92f9c2fa0882a2fd144be0f798a6c45b37e5b37: Status 404 returned error can't find the container with id f94e00891e3e9647d3753812c92f9c2fa0882a2fd144be0f798a6c45b37e5b37
Sep 07 11:46:29.260554 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.260523    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:46:29.260624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.260588    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:46:29.260660 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.260623    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:46:29.642359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.642260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" event={"ID":"d8864574-66ca-4c7c-a641-221ae21d1bae","Type":"ContainerStarted","Data":"17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"}
Sep 07 11:46:29.642359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.642324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" event={"ID":"d8864574-66ca-4c7c-a641-221ae21d1bae","Type":"ContainerStarted","Data":"f94e00891e3e9647d3753812c92f9c2fa0882a2fd144be0f798a6c45b37e5b37"}
Sep 07 11:46:29.661856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.661800    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podStartSLOduration=1.661781599 podStartE2EDuration="1.661781599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:46:29.660244771 +0000 UTC m=+1421.607595559" watchObservedRunningTime="2026-09-07 11:46:29.661781599 +0000 UTC m=+1421.609132385"
Sep 07 11:46:29.982983 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:29.982954    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:30.988537 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:30.988506    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:31.655129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:31.655087    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:31.656491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:31.656464    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:33.531335 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.531305    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"]
Sep 07 11:46:33.535294 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.535271    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.537667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.537649    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-2cfab4\""
Sep 07 11:46:33.537821 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.537651    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-2cfab4-dockercfg-zxhxq\""
Sep 07 11:46:33.546086 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.546063    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"]
Sep 07 11:46:33.582504 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.582466    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgwmg\" (UniqueName: \"kubernetes.io/projected/31f3382a-5790-46bb-919e-6723a3cb2d6d-kube-api-access-jgwmg\") pod \"payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") " pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.582644 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.582560    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f3382a-5790-46bb-919e-6723a3cb2d6d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") " pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.683342 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.683297    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgwmg\" (UniqueName: \"kubernetes.io/projected/31f3382a-5790-46bb-919e-6723a3cb2d6d-kube-api-access-jgwmg\") pod \"payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") " pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.683521 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.683361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f3382a-5790-46bb-919e-6723a3cb2d6d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") " pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.684163 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.684142    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f3382a-5790-46bb-919e-6723a3cb2d6d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") " pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.691990 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.691968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgwmg\" (UniqueName: \"kubernetes.io/projected/31f3382a-5790-46bb-919e-6723a3cb2d6d-kube-api-access-jgwmg\") pod \"payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") " pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:33.847759 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:33.847664    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:34.121530 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:34.121483    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"]
Sep 07 11:46:34.123687 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:46:34.123661    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f3382a_5790_46bb_919e_6723a3cb2d6d.slice/crio-14ea23ccefce82e1b73ff6e1d9c1491cf35dc2898ac3727d7391e1a02fa05026 WatchSource:0}: Error finding container 14ea23ccefce82e1b73ff6e1d9c1491cf35dc2898ac3727d7391e1a02fa05026: Status 404 returned error can't find the container with id 14ea23ccefce82e1b73ff6e1d9c1491cf35dc2898ac3727d7391e1a02fa05026
Sep 07 11:46:34.675280 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:34.674615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j" event={"ID":"31f3382a-5790-46bb-919e-6723a3cb2d6d","Type":"ContainerStarted","Data":"215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b"}
Sep 07 11:46:34.675280 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:34.674690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j" event={"ID":"31f3382a-5790-46bb-919e-6723a3cb2d6d","Type":"ContainerStarted","Data":"14ea23ccefce82e1b73ff6e1d9c1491cf35dc2898ac3727d7391e1a02fa05026"}
Sep 07 11:46:34.675280 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:34.674946    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:34.696026 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:34.695972    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j" podStartSLOduration=1.695955509 podStartE2EDuration="1.695955509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:46:34.693541212 +0000 UTC m=+1426.640891999" watchObservedRunningTime="2026-09-07 11:46:34.695955509 +0000 UTC m=+1426.643306293"
Sep 07 11:46:45.683258 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:45.683179    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:46:48.713035 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:48.713001    2560 scope.go:117] "RemoveContainer" containerID="5d8d2a91dcbed74670ef96e81a563c767dcd055d2fe8b60c27637d9b79fec856"
Sep 07 11:46:54.788079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:54.788044    2560 generic.go:356] "Generic (PLEG): container finished" podID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerID="17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73" exitCode=0
Sep 07 11:46:54.788610 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:54.788076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" event={"ID":"d8864574-66ca-4c7c-a641-221ae21d1bae","Type":"ContainerDied","Data":"17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"}
Sep 07 11:46:54.788693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:54.788605    2560 scope.go:117] "RemoveContainer" containerID="17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"
Sep 07 11:46:54.789849 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:54.789822    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:46:54.789914 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:54.789905    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:46:54.789951 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:54.789939    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:46:55.799550 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:55.799500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" event={"ID":"d8864574-66ca-4c7c-a641-221ae21d1bae","Type":"ContainerStarted","Data":"30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf"}
Sep 07 11:46:55.983492 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:55.983455    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:46:55.983685 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:55.983664    2560 patch_prober.go:28] interesting pod/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 07 11:46:55.983736 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:55.983714    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 07 11:46:56.983749 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:56.983680    2560 patch_prober.go:28] interesting pod/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 07 11:46:56.984129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:56.983756    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 07 11:46:57.982972 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:57.982931    2560 patch_prober.go:28] interesting pod/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 07 11:46:57.983161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:57.983016    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 07 11:46:58.983692 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:58.983657    2560 patch_prober.go:28] interesting pod/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 07 11:46:58.984087 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:58.983712    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 07 11:46:59.982894 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:59.982856    2560 patch_prober.go:28] interesting pod/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 07 11:46:59.983062 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:46:59.982916    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 07 11:47:00.983005 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:00.982968    2560 patch_prober.go:28] interesting pod/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 07 11:47:00.983463 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:00.983031    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 07 11:47:01.655023 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:01.654988    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:47:01.993214 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:01.993185    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:47:01.994366 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:01.994341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:47:03.129934 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.129895    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"]
Sep 07 11:47:03.133213 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.133192    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.135378 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.135357    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f57a16-dockercfg-q5g8p\""
Sep 07 11:47:03.135487 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.135386    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f57a16\""
Sep 07 11:47:03.143468 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.143444    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"]
Sep 07 11:47:03.278804 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.278769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ea45ee7-705a-430c-9bb8-55828eefee52-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.278978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.278816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfs24\" (UniqueName: \"kubernetes.io/projected/9ea45ee7-705a-430c-9bb8-55828eefee52-kube-api-access-bfs24\") pod \"payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.380291 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.380201    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ea45ee7-705a-430c-9bb8-55828eefee52-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.380291 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.380249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfs24\" (UniqueName: \"kubernetes.io/projected/9ea45ee7-705a-430c-9bb8-55828eefee52-kube-api-access-bfs24\") pod \"payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.380947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.380928    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ea45ee7-705a-430c-9bb8-55828eefee52-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.387682 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.387659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfs24\" (UniqueName: \"kubernetes.io/projected/9ea45ee7-705a-430c-9bb8-55828eefee52-kube-api-access-bfs24\") pod \"payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.443962 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.443935    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.714199 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.714175    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"]
Sep 07 11:47:03.716292 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:47:03.716264    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea45ee7_705a_430c_9bb8_55828eefee52.slice/crio-cc0709e15be66dde67fa041ac603d9bc90b86faa0d5c2f07c63ba5fedc145172 WatchSource:0}: Error finding container cc0709e15be66dde67fa041ac603d9bc90b86faa0d5c2f07c63ba5fedc145172: Status 404 returned error can't find the container with id cc0709e15be66dde67fa041ac603d9bc90b86faa0d5c2f07c63ba5fedc145172
Sep 07 11:47:03.856137 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.856100    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz" event={"ID":"9ea45ee7-705a-430c-9bb8-55828eefee52","Type":"ContainerStarted","Data":"cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e"}
Sep 07 11:47:03.856380 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.856157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz" event={"ID":"9ea45ee7-705a-430c-9bb8-55828eefee52","Type":"ContainerStarted","Data":"cc0709e15be66dde67fa041ac603d9bc90b86faa0d5c2f07c63ba5fedc145172"}
Sep 07 11:47:03.856380 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.856188    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:03.874342 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:03.874295    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz" podStartSLOduration=0.87427908 podStartE2EDuration="874.27908ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:03.871571913 +0000 UTC m=+1455.818922697" watchObservedRunningTime="2026-09-07 11:47:03.87427908 +0000 UTC m=+1455.821629915"
Sep 07 11:47:14.864098 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:14.864065    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:47:45.246139 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.246105    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"]
Sep 07 11:47:45.246570 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.246370    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j" podUID="31f3382a-5790-46bb-919e-6723a3cb2d6d" containerName="payload-processing" containerID="cri-o://215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b" gracePeriod=30
Sep 07 11:47:45.505017 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.504942    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:47:45.680122 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.680091    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f3382a-5790-46bb-919e-6723a3cb2d6d-plugins-config-volume\") pod \"31f3382a-5790-46bb-919e-6723a3cb2d6d\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") "
Sep 07 11:47:45.680371 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.680188    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgwmg\" (UniqueName: \"kubernetes.io/projected/31f3382a-5790-46bb-919e-6723a3cb2d6d-kube-api-access-jgwmg\") pod \"31f3382a-5790-46bb-919e-6723a3cb2d6d\" (UID: \"31f3382a-5790-46bb-919e-6723a3cb2d6d\") "
Sep 07 11:47:45.680518 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.680483    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31f3382a-5790-46bb-919e-6723a3cb2d6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31f3382a-5790-46bb-919e-6723a3cb2d6d" (UID: "31f3382a-5790-46bb-919e-6723a3cb2d6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:47:45.682183 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.682160    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f3382a-5790-46bb-919e-6723a3cb2d6d-kube-api-access-jgwmg" (OuterVolumeSpecName: "kube-api-access-jgwmg") pod "31f3382a-5790-46bb-919e-6723a3cb2d6d" (UID: "31f3382a-5790-46bb-919e-6723a3cb2d6d"). InnerVolumeSpecName "kube-api-access-jgwmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:47:45.781230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.781147    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgwmg\" (UniqueName: \"kubernetes.io/projected/31f3382a-5790-46bb-919e-6723a3cb2d6d-kube-api-access-jgwmg\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:45.781230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:45.781173    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31f3382a-5790-46bb-919e-6723a3cb2d6d-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:46.084233 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.084148    2560 generic.go:356] "Generic (PLEG): container finished" podID="31f3382a-5790-46bb-919e-6723a3cb2d6d" containerID="215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b" exitCode=0
Sep 07 11:47:46.084233 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.084196    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j" event={"ID":"31f3382a-5790-46bb-919e-6723a3cb2d6d","Type":"ContainerDied","Data":"215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b"}
Sep 07 11:47:46.084445 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.084235    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j" event={"ID":"31f3382a-5790-46bb-919e-6723a3cb2d6d","Type":"ContainerDied","Data":"14ea23ccefce82e1b73ff6e1d9c1491cf35dc2898ac3727d7391e1a02fa05026"}
Sep 07 11:47:46.084445 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.084255    2560 scope.go:117] "RemoveContainer" containerID="215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b"
Sep 07 11:47:46.084445 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.084197    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"
Sep 07 11:47:46.093332 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.093317    2560 scope.go:117] "RemoveContainer" containerID="215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b"
Sep 07 11:47:46.093599 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:47:46.093572    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b\": container with ID starting with 215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b not found: ID does not exist" containerID="215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b"
Sep 07 11:47:46.093663 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.093608    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b"} err="failed to get container status \"215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b\": rpc error: code = NotFound desc = could not find container \"215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b\": container with ID starting with 215689b1cfd2746e9d1c5049f338715e122a73e91f731ce740b613263db4f39b not found: ID does not exist"
Sep 07 11:47:46.106134 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.106108    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"]
Sep 07 11:47:46.108173 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.108151    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2cfab4-6b489798bc-prc9j"]
Sep 07 11:47:46.498723 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:46.498689    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31f3382a-5790-46bb-919e-6723a3cb2d6d" path="/var/lib/kubelet/pods/31f3382a-5790-46bb-919e-6723a3cb2d6d/volumes"
Sep 07 11:47:48.598832 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:48.598804    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:47:48.602778 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:48.602734    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:47:48.736715 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:48.736686    2560 scope.go:117] "RemoveContainer" containerID="2ae8ebba46144da2665f7d1a05a7f445604303ce26703b411280be3efb2a1801"
Sep 07 11:47:52.123984 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:52.123952    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"]
Sep 07 11:47:52.124409 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:52.124205    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy" containerID="cri-o://30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf" gracePeriod=30
Sep 07 11:47:57.384887 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.384825    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:47:57.488647 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488614    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-podinfo\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.488866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488655    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-credential-socket\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.488866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488706    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-certs\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.488866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488736    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-envoy\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.488866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488796    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cxqp\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-kube-api-access-4cxqp\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.489091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488873    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-socket\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.489091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488944    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-token\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.489091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.488988    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8864574-66ca-4c7c-a641-221ae21d1bae-istiod-ca-cert\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.489091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489011    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-data\") pod \"d8864574-66ca-4c7c-a641-221ae21d1bae\" (UID: \"d8864574-66ca-4c7c-a641-221ae21d1bae\") "
Sep 07 11:47:57.489091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489055    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:47:57.489365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489071    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:47:57.489365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489152    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:47:57.489365 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489306    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-data" (OuterVolumeSpecName: "istio-data") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:47:57.489503 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489485    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.489553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489510    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.489553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489523    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.489553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489550    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.489675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.489488    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8864574-66ca-4c7c-a641-221ae21d1bae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:47:57.491065 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.491016    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-kube-api-access-4cxqp" (OuterVolumeSpecName: "kube-api-access-4cxqp") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "kube-api-access-4cxqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:47:57.491178 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.491074    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:47:57.491264 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.491246    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:47:57.491347 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.491328    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8864574-66ca-4c7c-a641-221ae21d1bae" (UID: "d8864574-66ca-4c7c-a641-221ae21d1bae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:47:57.590907 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.590874    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.590907 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.590903    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8864574-66ca-4c7c-a641-221ae21d1bae-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.590907 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.590911    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.591144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.590920    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8864574-66ca-4c7c-a641-221ae21d1bae-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:57.591144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:57.590929    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cxqp\" (UniqueName: \"kubernetes.io/projected/d8864574-66ca-4c7c-a641-221ae21d1bae-kube-api-access-4cxqp\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:47:58.158798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.158762    2560 generic.go:356] "Generic (PLEG): container finished" podID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerID="30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf" exitCode=0
Sep 07 11:47:58.158798 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.158810    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"
Sep 07 11:47:58.159079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.158849    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" event={"ID":"d8864574-66ca-4c7c-a641-221ae21d1bae","Type":"ContainerDied","Data":"30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf"}
Sep 07 11:47:58.159079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.158911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q" event={"ID":"d8864574-66ca-4c7c-a641-221ae21d1bae","Type":"ContainerDied","Data":"f94e00891e3e9647d3753812c92f9c2fa0882a2fd144be0f798a6c45b37e5b37"}
Sep 07 11:47:58.159079 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.158943    2560 scope.go:117] "RemoveContainer" containerID="30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf"
Sep 07 11:47:58.168445 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.168428    2560 scope.go:117] "RemoveContainer" containerID="17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"
Sep 07 11:47:58.176458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.176439    2560 scope.go:117] "RemoveContainer" containerID="30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf"
Sep 07 11:47:58.176715 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:47:58.176696    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf\": container with ID starting with 30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf not found: ID does not exist" containerID="30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf"
Sep 07 11:47:58.176822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.176724    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf"} err="failed to get container status \"30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf\": rpc error: code = NotFound desc = could not find container \"30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf\": container with ID starting with 30764ae8e853c1954199763072fc04700a346e9e12f908e4c0baa44cbfbeedbf not found: ID does not exist"
Sep 07 11:47:58.176822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.176757    2560 scope.go:117] "RemoveContainer" containerID="17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"
Sep 07 11:47:58.177019 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:47:58.177002    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73\": container with ID starting with 17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73 not found: ID does not exist" containerID="17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"
Sep 07 11:47:58.177058 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.177026    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73"} err="failed to get container status \"17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73\": rpc error: code = NotFound desc = could not find container \"17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73\": container with ID starting with 17786c108563b1a9f1de537c8362d33d618958df249a1783114948ff18c30c73 not found: ID does not exist"
Sep 07 11:47:58.180679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.180659    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"]
Sep 07 11:47:58.184322 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.184301    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2cfab4-openshift-default-956f7bfcb-gj84q"]
Sep 07 11:47:58.497701 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:47:58.497667    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" path="/var/lib/kubelet/pods/d8864574-66ca-4c7c-a641-221ae21d1bae/volumes"
Sep 07 11:48:02.784234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784206    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"]
Sep 07 11:48:02.784628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784616    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy"
Sep 07 11:48:02.784675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784630    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy"
Sep 07 11:48:02.784675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784640    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy"
Sep 07 11:48:02.784675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784645    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy"
Sep 07 11:48:02.784675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784654    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f3382a-5790-46bb-919e-6723a3cb2d6d" containerName="payload-processing"
Sep 07 11:48:02.784675 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784661    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f3382a-5790-46bb-919e-6723a3cb2d6d" containerName="payload-processing"
Sep 07 11:48:02.784848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784753    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="31f3382a-5790-46bb-919e-6723a3cb2d6d" containerName="payload-processing"
Sep 07 11:48:02.784848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784766    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy"
Sep 07 11:48:02.784848 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.784771    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8864574-66ca-4c7c-a641-221ae21d1bae" containerName="istio-proxy"
Sep 07 11:48:02.789254 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.789236    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.791716 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.791689    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-984178-openshift-default-dockercfg-z76dg\""
Sep 07 11:48:02.799922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.799897    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"]
Sep 07 11:48:02.843109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-credential-socket\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843112    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-envoy\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843132    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnp4v\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-kube-api-access-jnp4v\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-data\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843271    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-socket\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-token\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-podinfo\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843361    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-certs\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.843555 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.843394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a317fb2-b5ba-4016-b7d6-fc387531b848-istiod-ca-cert\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944156 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-credential-socket\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944163    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-envoy\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnp4v\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-kube-api-access-jnp4v\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944232    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-data\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-socket\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944266    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-token\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-podinfo\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-certs\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944721 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944388    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a317fb2-b5ba-4016-b7d6-fc387531b848-istiod-ca-cert\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944721 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-credential-socket\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944721 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-socket\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944780    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-certs\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.944904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.944856    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-data\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.945099 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.945081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a317fb2-b5ba-4016-b7d6-fc387531b848-istiod-ca-cert\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.946481 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.946457    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-envoy\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.946721 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.946703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-podinfo\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.951734 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.951709    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnp4v\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-kube-api-access-jnp4v\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:02.951840 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:02.951813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-token\") pod \"e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") " pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:03.100315 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:03.100237    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:03.374965 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:03.374939    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"]
Sep 07 11:48:03.376905 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:48:03.376882    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a317fb2_b5ba_4016_b7d6_fc387531b848.slice/crio-a0a9bdce4fa407c9a430fc4508c6e3348efef003baeba9ed64a96658ddc54f4a WatchSource:0}: Error finding container a0a9bdce4fa407c9a430fc4508c6e3348efef003baeba9ed64a96658ddc54f4a: Status 404 returned error can't find the container with id a0a9bdce4fa407c9a430fc4508c6e3348efef003baeba9ed64a96658ddc54f4a
Sep 07 11:48:03.379722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:03.379690    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:48:03.379803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:03.379776    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:48:03.379844 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:03.379811    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:48:04.205794 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:04.205759    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" event={"ID":"6a317fb2-b5ba-4016-b7d6-fc387531b848","Type":"ContainerStarted","Data":"72576f518b3348a33b99621a8a0aa5a8edb1cbfe21738711304ed5ee980a3843"}
Sep 07 11:48:04.206162 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:04.205819    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" event={"ID":"6a317fb2-b5ba-4016-b7d6-fc387531b848","Type":"ContainerStarted","Data":"a0a9bdce4fa407c9a430fc4508c6e3348efef003baeba9ed64a96658ddc54f4a"}
Sep 07 11:48:04.225684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:04.225637    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podStartSLOduration=2.2256225609999998 podStartE2EDuration="2.225622561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:04.223604824 +0000 UTC m=+1516.170955609" watchObservedRunningTime="2026-09-07 11:48:04.225622561 +0000 UTC m=+1516.172973341"
Sep 07 11:48:05.101346 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:05.101309    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:05.106984 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:05.106959    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:05.211624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:05.211594    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:05.212444 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:05.212425    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:08.061317 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.061283    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"]
Sep 07 11:48:08.064968 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.064949    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.067029 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.066993    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-984178\""
Sep 07 11:48:08.067171 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.067155    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-984178-dockercfg-fp9f9\""
Sep 07 11:48:08.074670 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.074647    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"]
Sep 07 11:48:08.084617 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.084586    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"]
Sep 07 11:48:08.088571 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.088550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.097538 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.097516    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"]
Sep 07 11:48:08.191958 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.191923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/1729c6be-0661-46e4-8bb2-d4b3723146c3-kube-api-access-cbgjb\") pod \"payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.191958 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.191968    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz7dv\" (UniqueName: \"kubernetes.io/projected/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-kube-api-access-nz7dv\") pod \"payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.192234 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.192207    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.192301 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.192252    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1729c6be-0661-46e4-8bb2-d4b3723146c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.288396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.288363    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"]
Sep 07 11:48:08.288631 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.288604    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz" podUID="9ea45ee7-705a-430c-9bb8-55828eefee52" containerName="payload-pre-processing" containerID="cri-o://cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e" gracePeriod=30
Sep 07 11:48:08.293563 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.293535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz7dv\" (UniqueName: \"kubernetes.io/projected/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-kube-api-access-nz7dv\") pod \"payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.293706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.293640    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.293706 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.293663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1729c6be-0661-46e4-8bb2-d4b3723146c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.293889 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.293722    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/1729c6be-0661-46e4-8bb2-d4b3723146c3-kube-api-access-cbgjb\") pod \"payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.294698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.294671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.294876 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.294846    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1729c6be-0661-46e4-8bb2-d4b3723146c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.307131 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.307102    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz7dv\" (UniqueName: \"kubernetes.io/projected/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-kube-api-access-nz7dv\") pod \"payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.310114 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.310087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/1729c6be-0661-46e4-8bb2-d4b3723146c3-kube-api-access-cbgjb\") pod \"payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.377227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.377194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:08.400510 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.400480    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:08.571856 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.571824    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:48:08.656357 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.656331    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"]
Sep 07 11:48:08.658251 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:48:08.658216    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1729c6be_0661_46e4_8bb2_d4b3723146c3.slice/crio-ae73a0157544294eb14c65c581692bab70319d449d93b551a67ce5f37983c25e WatchSource:0}: Error finding container ae73a0157544294eb14c65c581692bab70319d449d93b551a67ce5f37983c25e: Status 404 returned error can't find the container with id ae73a0157544294eb14c65c581692bab70319d449d93b551a67ce5f37983c25e
Sep 07 11:48:08.693100 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.693069    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"]
Sep 07 11:48:08.694288 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:48:08.694260    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0dc8ae_c9b9_4975_910b_3150508bc1d5.slice/crio-31c5140d97f460952e8cb7ceacec065584f76c84226fe2e7aa325790587fadbe WatchSource:0}: Error finding container 31c5140d97f460952e8cb7ceacec065584f76c84226fe2e7aa325790587fadbe: Status 404 returned error can't find the container with id 31c5140d97f460952e8cb7ceacec065584f76c84226fe2e7aa325790587fadbe
Sep 07 11:48:08.697227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.697204    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ea45ee7-705a-430c-9bb8-55828eefee52-plugins-config-volume\") pod \"9ea45ee7-705a-430c-9bb8-55828eefee52\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") "
Sep 07 11:48:08.697363 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.697348    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfs24\" (UniqueName: \"kubernetes.io/projected/9ea45ee7-705a-430c-9bb8-55828eefee52-kube-api-access-bfs24\") pod \"9ea45ee7-705a-430c-9bb8-55828eefee52\" (UID: \"9ea45ee7-705a-430c-9bb8-55828eefee52\") "
Sep 07 11:48:08.697806 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.697541    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ea45ee7-705a-430c-9bb8-55828eefee52-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ea45ee7-705a-430c-9bb8-55828eefee52" (UID: "9ea45ee7-705a-430c-9bb8-55828eefee52"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:48:08.697890 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.697862    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ea45ee7-705a-430c-9bb8-55828eefee52-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:08.699698 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.699674    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea45ee7-705a-430c-9bb8-55828eefee52-kube-api-access-bfs24" (OuterVolumeSpecName: "kube-api-access-bfs24") pod "9ea45ee7-705a-430c-9bb8-55828eefee52" (UID: "9ea45ee7-705a-430c-9bb8-55828eefee52"). InnerVolumeSpecName "kube-api-access-bfs24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:48:08.798759 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:08.798717    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfs24\" (UniqueName: \"kubernetes.io/projected/9ea45ee7-705a-430c-9bb8-55828eefee52-kube-api-access-bfs24\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:09.236237 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.236201    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv" event={"ID":"bb0dc8ae-c9b9-4975-910b-3150508bc1d5","Type":"ContainerStarted","Data":"f3f99a11f69924e7e31a86e9621d87bde1ca3dce0b4b26fffa20b47b6f9961e0"}
Sep 07 11:48:09.236237 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.236253    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv" event={"ID":"bb0dc8ae-c9b9-4975-910b-3150508bc1d5","Type":"ContainerStarted","Data":"31c5140d97f460952e8cb7ceacec065584f76c84226fe2e7aa325790587fadbe"}
Sep 07 11:48:09.236731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.236295    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:09.238090 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.238067    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t" event={"ID":"1729c6be-0661-46e4-8bb2-d4b3723146c3","Type":"ContainerStarted","Data":"0119a0817e23cad126487b669d410d254a8db04d01c7bcceaabfdb2daaa82f1f"}
Sep 07 11:48:09.238226 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.238106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t" event={"ID":"1729c6be-0661-46e4-8bb2-d4b3723146c3","Type":"ContainerStarted","Data":"ae73a0157544294eb14c65c581692bab70319d449d93b551a67ce5f37983c25e"}
Sep 07 11:48:09.238226 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.238132    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:09.239522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.239498    2560 generic.go:356] "Generic (PLEG): container finished" podID="9ea45ee7-705a-430c-9bb8-55828eefee52" containerID="cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e" exitCode=0
Sep 07 11:48:09.239618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.239546    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"
Sep 07 11:48:09.239618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.239583    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz" event={"ID":"9ea45ee7-705a-430c-9bb8-55828eefee52","Type":"ContainerDied","Data":"cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e"}
Sep 07 11:48:09.239703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.239634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz" event={"ID":"9ea45ee7-705a-430c-9bb8-55828eefee52","Type":"ContainerDied","Data":"cc0709e15be66dde67fa041ac603d9bc90b86faa0d5c2f07c63ba5fedc145172"}
Sep 07 11:48:09.239703 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.239659    2560 scope.go:117] "RemoveContainer" containerID="cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e"
Sep 07 11:48:09.249385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.249370    2560 scope.go:117] "RemoveContainer" containerID="cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e"
Sep 07 11:48:09.249640 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:48:09.249619    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e\": container with ID starting with cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e not found: ID does not exist" containerID="cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e"
Sep 07 11:48:09.249704 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.249652    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e"} err="failed to get container status \"cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e\": rpc error: code = NotFound desc = could not find container \"cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e\": container with ID starting with cd2c864aa9f94fdbcb25600439a53455cbdb2e74b8a1a6a258bdb538bb99554e not found: ID does not exist"
Sep 07 11:48:09.253348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.253274    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv" podStartSLOduration=1.253258936 podStartE2EDuration="1.253258936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:09.252609884 +0000 UTC m=+1521.199960670" watchObservedRunningTime="2026-09-07 11:48:09.253258936 +0000 UTC m=+1521.200609722"
Sep 07 11:48:09.264331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.264311    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"]
Sep 07 11:48:09.269966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.269945    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f57a16-64c6fb8f94-dkbkz"]
Sep 07 11:48:09.281622 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:09.281577    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t" podStartSLOduration=1.281561371 podStartE2EDuration="1.281561371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:09.281159075 +0000 UTC m=+1521.228509858" watchObservedRunningTime="2026-09-07 11:48:09.281561371 +0000 UTC m=+1521.228912156"
Sep 07 11:48:10.497321 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:10.497290    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ea45ee7-705a-430c-9bb8-55828eefee52" path="/var/lib/kubelet/pods/9ea45ee7-705a-430c-9bb8-55828eefee52/volumes"
Sep 07 11:48:20.247759 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:20.247708    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:48:20.248311 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:20.247843    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:48:25.927951 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:25.927863    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"]
Sep 07 11:48:25.929936 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:25.929913    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-0c816d-29813025-q79m7"]
Sep 07 11:48:25.951396 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:25.951366    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"]
Sep 07 11:48:25.951648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:25.951621    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj" podUID="1e848345-8956-476f-9ee7-8e0ff6a4d692" containerName="payload-processing" containerID="cri-o://c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107" gracePeriod=30
Sep 07 11:48:26.213498 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.213474    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:48:26.280214 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.280183    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8c9h\" (UniqueName: \"kubernetes.io/projected/1e848345-8956-476f-9ee7-8e0ff6a4d692-kube-api-access-x8c9h\") pod \"1e848345-8956-476f-9ee7-8e0ff6a4d692\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") "
Sep 07 11:48:26.280343 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.280233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e848345-8956-476f-9ee7-8e0ff6a4d692-plugins-config-volume\") pod \"1e848345-8956-476f-9ee7-8e0ff6a4d692\" (UID: \"1e848345-8956-476f-9ee7-8e0ff6a4d692\") "
Sep 07 11:48:26.280628 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.280611    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e848345-8956-476f-9ee7-8e0ff6a4d692-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e848345-8956-476f-9ee7-8e0ff6a4d692" (UID: "1e848345-8956-476f-9ee7-8e0ff6a4d692"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:48:26.282257 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.282226    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e848345-8956-476f-9ee7-8e0ff6a4d692-kube-api-access-x8c9h" (OuterVolumeSpecName: "kube-api-access-x8c9h") pod "1e848345-8956-476f-9ee7-8e0ff6a4d692" (UID: "1e848345-8956-476f-9ee7-8e0ff6a4d692"). InnerVolumeSpecName "kube-api-access-x8c9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:48:26.354790 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.354731    2560 generic.go:356] "Generic (PLEG): container finished" podID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerID="72576f518b3348a33b99621a8a0aa5a8edb1cbfe21738711304ed5ee980a3843" exitCode=0
Sep 07 11:48:26.354956 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.354814    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" event={"ID":"6a317fb2-b5ba-4016-b7d6-fc387531b848","Type":"ContainerDied","Data":"72576f518b3348a33b99621a8a0aa5a8edb1cbfe21738711304ed5ee980a3843"}
Sep 07 11:48:26.355368 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.355347    2560 scope.go:117] "RemoveContainer" containerID="72576f518b3348a33b99621a8a0aa5a8edb1cbfe21738711304ed5ee980a3843"
Sep 07 11:48:26.356212 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356190    2560 generic.go:356] "Generic (PLEG): container finished" podID="1e848345-8956-476f-9ee7-8e0ff6a4d692" containerID="c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107" exitCode=0
Sep 07 11:48:26.356302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356252    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj" event={"ID":"1e848345-8956-476f-9ee7-8e0ff6a4d692","Type":"ContainerDied","Data":"c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107"}
Sep 07 11:48:26.356302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356281    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"
Sep 07 11:48:26.356302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356298    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj" event={"ID":"1e848345-8956-476f-9ee7-8e0ff6a4d692","Type":"ContainerDied","Data":"1c93d037554c69b0b87bd9f08515c9550dbcb9693e44b41d611dbfbaeea9c4a6"}
Sep 07 11:48:26.356406 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356328    2560 scope.go:117] "RemoveContainer" containerID="c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107"
Sep 07 11:48:26.356504 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356463    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:48:26.356611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356541    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:48:26.356611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.356576    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 07 11:48:26.368089 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.368068    2560 scope.go:117] "RemoveContainer" containerID="c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107"
Sep 07 11:48:26.368358 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:48:26.368337    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107\": container with ID starting with c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107 not found: ID does not exist" containerID="c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107"
Sep 07 11:48:26.368416 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.368368    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107"} err="failed to get container status \"c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107\": rpc error: code = NotFound desc = could not find container \"c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107\": container with ID starting with c932dcdee405487607b1184390f8c0081efbabef54bf9e8989fdcff43e912107 not found: ID does not exist"
Sep 07 11:48:26.381533 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.381508    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8c9h\" (UniqueName: \"kubernetes.io/projected/1e848345-8956-476f-9ee7-8e0ff6a4d692-kube-api-access-x8c9h\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:26.381667 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.381538    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e848345-8956-476f-9ee7-8e0ff6a4d692-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:26.388660 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.388635    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"]
Sep 07 11:48:26.392664 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.392643    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0c816d-74486dfcb9-pq9tj"]
Sep 07 11:48:26.498600 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.498565    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e848345-8956-476f-9ee7-8e0ff6a4d692" path="/var/lib/kubelet/pods/1e848345-8956-476f-9ee7-8e0ff6a4d692/volumes"
Sep 07 11:48:26.499003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:26.498981    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c4b82e-759d-439f-9103-902521b02d4d" path="/var/lib/kubelet/pods/24c4b82e-759d-439f-9103-902521b02d4d/volumes"
Sep 07 11:48:27.364418 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:27.364384    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" event={"ID":"6a317fb2-b5ba-4016-b7d6-fc387531b848","Type":"ContainerStarted","Data":"77d64e7fcd408c376166e73edcf89cd1717c83db6dc7864681b9f2ffd7ce848c"}
Sep 07 11:48:28.101334 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:28.101287    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:28.101554 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:28.101446    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 07 11:48:28.101554 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:28.101532    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 07 11:48:29.101023 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:29.100984    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 07 11:48:29.101632 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:29.101598    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 07 11:48:30.100800 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:30.100762    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 07 11:48:30.101085 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:30.101054    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 07 11:48:31.101592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:31.101544    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 07 11:48:31.102106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:31.101629    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 07 11:48:32.101404 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:32.101362    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 07 11:48:32.101619 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:32.101437    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 07 11:48:33.105733 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:33.105694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:33.106188 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:33.106018    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:33.106816 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:33.106800    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:48:37.794930 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.794895    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"]
Sep 07 11:48:37.795348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.795334    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e848345-8956-476f-9ee7-8e0ff6a4d692" containerName="payload-processing"
Sep 07 11:48:37.795398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.795350    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e848345-8956-476f-9ee7-8e0ff6a4d692" containerName="payload-processing"
Sep 07 11:48:37.795398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.795375    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea45ee7-705a-430c-9bb8-55828eefee52" containerName="payload-pre-processing"
Sep 07 11:48:37.795398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.795381    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea45ee7-705a-430c-9bb8-55828eefee52" containerName="payload-pre-processing"
Sep 07 11:48:37.795500 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.795447    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e848345-8956-476f-9ee7-8e0ff6a4d692" containerName="payload-processing"
Sep 07 11:48:37.795500 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.795456    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ea45ee7-705a-430c-9bb8-55828eefee52" containerName="payload-pre-processing"
Sep 07 11:48:37.798959 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.798925    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:37.801050 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.801025    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f866d4\""
Sep 07 11:48:37.801553 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.801533    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f866d4-dockercfg-jw8rf\""
Sep 07 11:48:37.812232 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.812201    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"]
Sep 07 11:48:37.902997 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.902964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f57016-b93c-4550-b03e-d07ce9b2c731-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:37.903170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:37.903058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfs7\" (UniqueName: \"kubernetes.io/projected/d2f57016-b93c-4550-b03e-d07ce9b2c731-kube-api-access-djfs7\") pod \"payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:38.004496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.004460    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f57016-b93c-4550-b03e-d07ce9b2c731-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:38.004682 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.004534    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfs7\" (UniqueName: \"kubernetes.io/projected/d2f57016-b93c-4550-b03e-d07ce9b2c731-kube-api-access-djfs7\") pod \"payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:38.005188 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.005169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f57016-b93c-4550-b03e-d07ce9b2c731-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:38.012135 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.012099    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfs7\" (UniqueName: \"kubernetes.io/projected/d2f57016-b93c-4550-b03e-d07ce9b2c731-kube-api-access-djfs7\") pod \"payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:38.122908 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.122818    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:38.397331 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.397305    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"]
Sep 07 11:48:38.400358 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:48:38.400324    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f57016_b93c_4550_b03e_d07ce9b2c731.slice/crio-a991e78600e0c1603c14080b416d44f2657bc9913ea3053e439fc59eddcae7af WatchSource:0}: Error finding container a991e78600e0c1603c14080b416d44f2657bc9913ea3053e439fc59eddcae7af: Status 404 returned error can't find the container with id a991e78600e0c1603c14080b416d44f2657bc9913ea3053e439fc59eddcae7af
Sep 07 11:48:38.451805 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:38.451769    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k" event={"ID":"d2f57016-b93c-4550-b03e-d07ce9b2c731","Type":"ContainerStarted","Data":"a991e78600e0c1603c14080b416d44f2657bc9913ea3053e439fc59eddcae7af"}
Sep 07 11:48:39.465376 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:39.465338    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k" event={"ID":"d2f57016-b93c-4550-b03e-d07ce9b2c731","Type":"ContainerStarted","Data":"db1724d1dd50c813922882013e1e627b854c9610a60f1e075b0eeec4b308eb4f"}
Sep 07 11:48:39.465790 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:39.465462    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:39.482760 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:39.482695    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k" podStartSLOduration=2.482682082 podStartE2EDuration="2.482682082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:39.480960912 +0000 UTC m=+1551.428311696" watchObservedRunningTime="2026-09-07 11:48:39.482682082 +0000 UTC m=+1551.430032867"
Sep 07 11:48:44.341003 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.340972    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"]
Sep 07 11:48:44.341456 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.341224    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx" podUID="39db941a-fcc2-4b0c-a034-98e1e84b1bf9" containerName="payload-processing" containerID="cri-o://c8c3e00ea43e36c6ea3809574bdaf535ccca06629a3c279af06a9f86ee1a6505" gracePeriod=30
Sep 07 11:48:44.495612 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.495580    2560 generic.go:356] "Generic (PLEG): container finished" podID="39db941a-fcc2-4b0c-a034-98e1e84b1bf9" containerID="c8c3e00ea43e36c6ea3809574bdaf535ccca06629a3c279af06a9f86ee1a6505" exitCode=0
Sep 07 11:48:44.499204 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.499177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx" event={"ID":"39db941a-fcc2-4b0c-a034-98e1e84b1bf9","Type":"ContainerDied","Data":"c8c3e00ea43e36c6ea3809574bdaf535ccca06629a3c279af06a9f86ee1a6505"}
Sep 07 11:48:44.589245 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.589218    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:48:44.671569 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.671480    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9ctl\" (UniqueName: \"kubernetes.io/projected/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-kube-api-access-p9ctl\") pod \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") "
Sep 07 11:48:44.671766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.671588    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-plugins-config-volume\") pod \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\" (UID: \"39db941a-fcc2-4b0c-a034-98e1e84b1bf9\") "
Sep 07 11:48:44.672261 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.672229    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39db941a-fcc2-4b0c-a034-98e1e84b1bf9" (UID: "39db941a-fcc2-4b0c-a034-98e1e84b1bf9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:48:44.674460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.674424    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-kube-api-access-p9ctl" (OuterVolumeSpecName: "kube-api-access-p9ctl") pod "39db941a-fcc2-4b0c-a034-98e1e84b1bf9" (UID: "39db941a-fcc2-4b0c-a034-98e1e84b1bf9"). InnerVolumeSpecName "kube-api-access-p9ctl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:48:44.773247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.773214    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9ctl\" (UniqueName: \"kubernetes.io/projected/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-kube-api-access-p9ctl\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:44.773247 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:44.773241    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39db941a-fcc2-4b0c-a034-98e1e84b1bf9-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:45.506161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:45.506116    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx" event={"ID":"39db941a-fcc2-4b0c-a034-98e1e84b1bf9","Type":"ContainerDied","Data":"c9ccfd6dbb5f7c1bda1cd24978fd007bc0fe051f0c723906d9d6b2747f41890d"}
Sep 07 11:48:45.506645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:45.506193    2560 scope.go:117] "RemoveContainer" containerID="c8c3e00ea43e36c6ea3809574bdaf535ccca06629a3c279af06a9f86ee1a6505"
Sep 07 11:48:45.506645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:45.506367    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"
Sep 07 11:48:45.532151 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:45.532124    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"]
Sep 07 11:48:45.539185 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:45.539151    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2195e0-696897cb9f-5tjdx"]
Sep 07 11:48:46.497908 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:46.497875    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39db941a-fcc2-4b0c-a034-98e1e84b1bf9" path="/var/lib/kubelet/pods/39db941a-fcc2-4b0c-a034-98e1e84b1bf9/volumes"
Sep 07 11:48:50.474526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:50.474497    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:48:51.326162 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:51.326131    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"]
Sep 07 11:48:51.326469 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:51.326353    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" containerID="cri-o://a3eb13117c263d95fefa2ad09b62aae77106a0d1374162e1ee0f8a05aa7a8311" gracePeriod=30
Sep 07 11:48:53.014000 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:53.013959    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 11:48:53.014382 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:53.014017    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 11:48:56.576777 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.576731    2560 generic.go:356] "Generic (PLEG): container finished" podID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerID="a3eb13117c263d95fefa2ad09b62aae77106a0d1374162e1ee0f8a05aa7a8311" exitCode=0
Sep 07 11:48:56.577096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.576777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" event={"ID":"41ca5647-73aa-4ec7-ba83-8ccd64805ae0","Type":"ContainerDied","Data":"a3eb13117c263d95fefa2ad09b62aae77106a0d1374162e1ee0f8a05aa7a8311"}
Sep 07 11:48:56.577096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.576837    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz" event={"ID":"41ca5647-73aa-4ec7-ba83-8ccd64805ae0","Type":"ContainerDied","Data":"306c24655c4303aff0a6371c93d1e5c89339b997182fb28b5d18af037f4d36c2"}
Sep 07 11:48:56.577096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.576856    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="306c24655c4303aff0a6371c93d1e5c89339b997182fb28b5d18af037f4d36c2"
Sep 07 11:48:56.577096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.576869    2560 scope.go:117] "RemoveContainer" containerID="12f529cbb294c9a2a7d02f0c89b21e8468593f9e162e087dd54656786be6870e"
Sep 07 11:48:56.585665 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.585606    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:48:56.691184 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691096    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5k2n\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-kube-api-access-z5k2n\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691184 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-data\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691184 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691163    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-podinfo\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691203    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-token\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691220    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-credential-socket\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691260    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-envoy\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691318    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-socket\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691394    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istiod-ca-cert\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691404    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-data" (OuterVolumeSpecName: "istio-data") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:48:56.691454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691440    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-certs\") pod \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\" (UID: \"41ca5647-73aa-4ec7-ba83-8ccd64805ae0\") "
Sep 07 11:48:56.691859 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691542    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:48:56.691859 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691632    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:48:56.691859 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691721    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:48:56.691859 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691799    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:48:56.692077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691866    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.692077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691883    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.692077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691898    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.692077 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.691911    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.693406 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.693376    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-kube-api-access-z5k2n" (OuterVolumeSpecName: "kube-api-access-z5k2n") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "kube-api-access-z5k2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:48:56.693512 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.693449    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-token" (OuterVolumeSpecName: "istio-token") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:48:56.693735 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.693714    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:48:56.693826 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.693717    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41ca5647-73aa-4ec7-ba83-8ccd64805ae0" (UID: "41ca5647-73aa-4ec7-ba83-8ccd64805ae0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:48:56.793152 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.793133    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.793152 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.793152    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.793295 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.793161    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.793295 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.793170    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5k2n\" (UniqueName: \"kubernetes.io/projected/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-kube-api-access-z5k2n\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:56.793295 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:56.793178    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ca5647-73aa-4ec7-ba83-8ccd64805ae0-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:48:57.583265 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:57.583235    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"
Sep 07 11:48:57.621406 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:57.621371    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"]
Sep 07 11:48:57.625052 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:57.625028    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2195e0-openshift-default-fdf554f88-8znhz"]
Sep 07 11:48:58.497548 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:48:58.497515    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" path="/var/lib/kubelet/pods/41ca5647-73aa-4ec7-ba83-8ccd64805ae0/volumes"
Sep 07 11:49:06.545752 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:06.545707    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pm4dl_bbc264ca-820b-46cc-974f-e6feceb75d72/payload-processing/0.log"
Sep 07 11:49:06.666207 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:06.666144    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k_d2f57016-b93c-4550-b03e-d07ce9b2c731/payload-processing/0.log"
Sep 07 11:49:34.615956 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:34.615923    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv_bb0dc8ae-c9b9-4975-910b-3150508bc1d5/payload-processing/0.log"
Sep 07 11:49:34.763579 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:34.763543    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pm4dl_bbc264ca-820b-46cc-974f-e6feceb75d72/payload-processing/0.log"
Sep 07 11:49:34.892115 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:34.892033    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k_d2f57016-b93c-4550-b03e-d07ce9b2c731/payload-processing/0.log"
Sep 07 11:49:40.327456 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.327422    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"]
Sep 07 11:49:40.327896 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.327880    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy"
Sep 07 11:49:40.327947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.327898    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy"
Sep 07 11:49:40.327947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.327923    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39db941a-fcc2-4b0c-a034-98e1e84b1bf9" containerName="payload-processing"
Sep 07 11:49:40.327947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.327929    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39db941a-fcc2-4b0c-a034-98e1e84b1bf9" containerName="payload-processing"
Sep 07 11:49:40.328290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.327997    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy"
Sep 07 11:49:40.328290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.328006    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39db941a-fcc2-4b0c-a034-98e1e84b1bf9" containerName="payload-processing"
Sep 07 11:49:40.328290 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.328014    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy"
Sep 07 11:49:40.331452 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.331434    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.333551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.333534    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-55c3cf\""
Sep 07 11:49:40.333656 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.333537    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-55c3cf-dockercfg-5fsfp\""
Sep 07 11:49:40.342053 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.342027    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"]
Sep 07 11:49:40.402988 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.402952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxtl6\" (UniqueName: \"kubernetes.io/projected/8702ebe8-4b81-4161-8285-13b163c62943-kube-api-access-qxtl6\") pod \"payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.403136 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.403031    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8702ebe8-4b81-4161-8285-13b163c62943-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.503529 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.503498    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxtl6\" (UniqueName: \"kubernetes.io/projected/8702ebe8-4b81-4161-8285-13b163c62943-kube-api-access-qxtl6\") pod \"payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.503693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.503577    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8702ebe8-4b81-4161-8285-13b163c62943-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.504198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.504178    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8702ebe8-4b81-4161-8285-13b163c62943-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.511261 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.511240    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxtl6\" (UniqueName: \"kubernetes.io/projected/8702ebe8-4b81-4161-8285-13b163c62943-kube-api-access-qxtl6\") pod \"payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.642592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.642493    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:40.913385 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:40.913361    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"]
Sep 07 11:49:40.915222 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:49:40.915194    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8702ebe8_4b81_4161_8285_13b163c62943.slice/crio-0a6de82779a0de8386ba654aa6c1e4b84d6f69a4ac649e1d134d8dc278babd15 WatchSource:0}: Error finding container 0a6de82779a0de8386ba654aa6c1e4b84d6f69a4ac649e1d134d8dc278babd15: Status 404 returned error can't find the container with id 0a6de82779a0de8386ba654aa6c1e4b84d6f69a4ac649e1d134d8dc278babd15
Sep 07 11:49:41.816673 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:41.816634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5" event={"ID":"8702ebe8-4b81-4161-8285-13b163c62943","Type":"ContainerStarted","Data":"cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96"}
Sep 07 11:49:41.817129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:41.816691    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5" event={"ID":"8702ebe8-4b81-4161-8285-13b163c62943","Type":"ContainerStarted","Data":"0a6de82779a0de8386ba654aa6c1e4b84d6f69a4ac649e1d134d8dc278babd15"}
Sep 07 11:49:41.817129 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:41.816763    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:49:41.847852 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:41.847797    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5" podStartSLOduration=1.847781027 podStartE2EDuration="1.847781027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:49:41.847419758 +0000 UTC m=+1613.794770542" watchObservedRunningTime="2026-09-07 11:49:41.847781027 +0000 UTC m=+1613.795131811"
Sep 07 11:49:48.786167 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:48.786137    2560 scope.go:117] "RemoveContainer" containerID="a3eb13117c263d95fefa2ad09b62aae77106a0d1374162e1ee0f8a05aa7a8311"
Sep 07 11:49:52.825432 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:49:52.825398    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:50:04.137456 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.137424    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"]
Sep 07 11:50:04.137980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.137897    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy"
Sep 07 11:50:04.137980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.137910    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ca5647-73aa-4ec7-ba83-8ccd64805ae0" containerName="istio-proxy"
Sep 07 11:50:04.141044 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.141027    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:04.143062 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.143028    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:50:04.148037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.148014    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"]
Sep 07 11:50:04.237183 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.237142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e66a86-b85c-4e30-b66a-b54d2395b07b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx\" (UID: \"17e66a86-b85c-4e30-b66a-b54d2395b07b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:04.337945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.337912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e66a86-b85c-4e30-b66a-b54d2395b07b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx\" (UID: \"17e66a86-b85c-4e30-b66a-b54d2395b07b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:04.338526 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.338507    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e66a86-b85c-4e30-b66a-b54d2395b07b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx\" (UID: \"17e66a86-b85c-4e30-b66a-b54d2395b07b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:04.452813 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.452709    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:04.719959 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.719935    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"]
Sep 07 11:50:04.722111 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:50:04.722086    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e66a86_b85c_4e30_b66a_b54d2395b07b.slice/crio-e65d11eb880bf1ea54b3daf57a5554485a5aec4ae83bc3fa4718dea2328082bc WatchSource:0}: Error finding container e65d11eb880bf1ea54b3daf57a5554485a5aec4ae83bc3fa4718dea2328082bc: Status 404 returned error can't find the container with id e65d11eb880bf1ea54b3daf57a5554485a5aec4ae83bc3fa4718dea2328082bc
Sep 07 11:50:04.940348 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.940321    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx" event={"ID":"17e66a86-b85c-4e30-b66a-b54d2395b07b","Type":"ContainerStarted","Data":"97ae3c8d8b52b08d8327cd02bd426f226d621114234ad7239ec830b829b4a597"}
Sep 07 11:50:04.940478 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.940370    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx" event={"ID":"17e66a86-b85c-4e30-b66a-b54d2395b07b","Type":"ContainerStarted","Data":"e65d11eb880bf1ea54b3daf57a5554485a5aec4ae83bc3fa4718dea2328082bc"}
Sep 07 11:50:04.955132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:04.955090    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx" podStartSLOduration=0.955075784 podStartE2EDuration="955.075784ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:50:04.953456076 +0000 UTC m=+1636.900806860" watchObservedRunningTime="2026-09-07 11:50:04.955075784 +0000 UTC m=+1636.902426568"
Sep 07 11:50:05.947606 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:05.947571    2560 generic.go:356] "Generic (PLEG): container finished" podID="17e66a86-b85c-4e30-b66a-b54d2395b07b" containerID="97ae3c8d8b52b08d8327cd02bd426f226d621114234ad7239ec830b829b4a597" exitCode=0
Sep 07 11:50:05.948038 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:05.947659    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx" event={"ID":"17e66a86-b85c-4e30-b66a-b54d2395b07b","Type":"ContainerDied","Data":"97ae3c8d8b52b08d8327cd02bd426f226d621114234ad7239ec830b829b4a597"}
Sep 07 11:50:07.087239 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.087213    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:07.168640 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.168608    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e66a86-b85c-4e30-b66a-b54d2395b07b-maas-api-service-ca\") pod \"17e66a86-b85c-4e30-b66a-b54d2395b07b\" (UID: \"17e66a86-b85c-4e30-b66a-b54d2395b07b\") "
Sep 07 11:50:07.168946 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.168922    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17e66a86-b85c-4e30-b66a-b54d2395b07b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17e66a86-b85c-4e30-b66a-b54d2395b07b" (UID: "17e66a86-b85c-4e30-b66a-b54d2395b07b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:50:07.169037 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.169025    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e66a86-b85c-4e30-b66a-b54d2395b07b-maas-api-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:07.959085 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.959056    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"
Sep 07 11:50:07.959262 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.959087    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx" event={"ID":"17e66a86-b85c-4e30-b66a-b54d2395b07b","Type":"ContainerDied","Data":"e65d11eb880bf1ea54b3daf57a5554485a5aec4ae83bc3fa4718dea2328082bc"}
Sep 07 11:50:07.959262 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:07.959133    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e65d11eb880bf1ea54b3daf57a5554485a5aec4ae83bc3fa4718dea2328082bc"
Sep 07 11:50:10.082167 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.082131    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"]
Sep 07 11:50:10.082571 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.082348    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5" podUID="8702ebe8-4b81-4161-8285-13b163c62943" containerName="payload-pre-processing" containerID="cri-o://cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96" gracePeriod=30
Sep 07 11:50:10.341393 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.341327    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:50:10.368209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.368175    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"]
Sep 07 11:50:10.370873 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.370846    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-55c3cf-6z8tx"]
Sep 07 11:50:10.500931 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.500904    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8702ebe8-4b81-4161-8285-13b163c62943-plugins-config-volume\") pod \"8702ebe8-4b81-4161-8285-13b163c62943\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") "
Sep 07 11:50:10.501097 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.500992    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxtl6\" (UniqueName: \"kubernetes.io/projected/8702ebe8-4b81-4161-8285-13b163c62943-kube-api-access-qxtl6\") pod \"8702ebe8-4b81-4161-8285-13b163c62943\" (UID: \"8702ebe8-4b81-4161-8285-13b163c62943\") "
Sep 07 11:50:10.501097 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.500909    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17e66a86-b85c-4e30-b66a-b54d2395b07b" path="/var/lib/kubelet/pods/17e66a86-b85c-4e30-b66a-b54d2395b07b/volumes"
Sep 07 11:50:10.501221 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.501195    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8702ebe8-4b81-4161-8285-13b163c62943-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8702ebe8-4b81-4161-8285-13b163c62943" (UID: "8702ebe8-4b81-4161-8285-13b163c62943"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:50:10.503070 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.503035    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8702ebe8-4b81-4161-8285-13b163c62943-kube-api-access-qxtl6" (OuterVolumeSpecName: "kube-api-access-qxtl6") pod "8702ebe8-4b81-4161-8285-13b163c62943" (UID: "8702ebe8-4b81-4161-8285-13b163c62943"). InnerVolumeSpecName "kube-api-access-qxtl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:50:10.602421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.602326    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxtl6\" (UniqueName: \"kubernetes.io/projected/8702ebe8-4b81-4161-8285-13b163c62943-kube-api-access-qxtl6\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:10.602421 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.602365    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8702ebe8-4b81-4161-8285-13b163c62943-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:10.974904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.974807    2560 generic.go:356] "Generic (PLEG): container finished" podID="8702ebe8-4b81-4161-8285-13b163c62943" containerID="cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96" exitCode=0
Sep 07 11:50:10.974904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.974847    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"
Sep 07 11:50:10.974904 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.974897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5" event={"ID":"8702ebe8-4b81-4161-8285-13b163c62943","Type":"ContainerDied","Data":"cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96"}
Sep 07 11:50:10.975150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.974944    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5" event={"ID":"8702ebe8-4b81-4161-8285-13b163c62943","Type":"ContainerDied","Data":"0a6de82779a0de8386ba654aa6c1e4b84d6f69a4ac649e1d134d8dc278babd15"}
Sep 07 11:50:10.975150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.974968    2560 scope.go:117] "RemoveContainer" containerID="cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96"
Sep 07 11:50:10.984043 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.984025    2560 scope.go:117] "RemoveContainer" containerID="cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96"
Sep 07 11:50:10.984311 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:50:10.984292    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96\": container with ID starting with cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96 not found: ID does not exist" containerID="cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96"
Sep 07 11:50:10.984381 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.984322    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96"} err="failed to get container status \"cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96\": rpc error: code = NotFound desc = could not find container \"cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96\": container with ID starting with cdf2eca5d4d6c2b15365bbd3c48c613d374f91c4c3a8e660aa7d0727a31fac96 not found: ID does not exist"
Sep 07 11:50:10.995176 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.995153    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"]
Sep 07 11:50:10.999586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:10.999566    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-55c3cf-d976b7c46-qd7w5"]
Sep 07 11:50:12.503099 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:12.503064    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8702ebe8-4b81-4161-8285-13b163c62943" path="/var/lib/kubelet/pods/8702ebe8-4b81-4161-8285-13b163c62943/volumes"
Sep 07 11:50:36.886428 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:36.886391    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"]
Sep 07 11:50:36.887051 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:36.886630    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv" podUID="bb0dc8ae-c9b9-4975-910b-3150508bc1d5" containerName="payload-processing" containerID="cri-o://f3f99a11f69924e7e31a86e9621d87bde1ca3dce0b4b26fffa20b47b6f9961e0" gracePeriod=30
Sep 07 11:50:36.892846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:36.892768    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"]
Sep 07 11:50:36.892995 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:36.892970    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t" podUID="1729c6be-0661-46e4-8bb2-d4b3723146c3" containerName="payload-pre-processing" containerID="cri-o://0119a0817e23cad126487b669d410d254a8db04d01c7bcceaabfdb2daaa82f1f" gracePeriod=30
Sep 07 11:50:37.120489 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.116990    2560 generic.go:356] "Generic (PLEG): container finished" podID="bb0dc8ae-c9b9-4975-910b-3150508bc1d5" containerID="f3f99a11f69924e7e31a86e9621d87bde1ca3dce0b4b26fffa20b47b6f9961e0" exitCode=0
Sep 07 11:50:37.120489 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.117124    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv" event={"ID":"bb0dc8ae-c9b9-4975-910b-3150508bc1d5","Type":"ContainerDied","Data":"f3f99a11f69924e7e31a86e9621d87bde1ca3dce0b4b26fffa20b47b6f9961e0"}
Sep 07 11:50:37.120489 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.120128    2560 generic.go:356] "Generic (PLEG): container finished" podID="1729c6be-0661-46e4-8bb2-d4b3723146c3" containerID="0119a0817e23cad126487b669d410d254a8db04d01c7bcceaabfdb2daaa82f1f" exitCode=0
Sep 07 11:50:37.120489 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.120206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t" event={"ID":"1729c6be-0661-46e4-8bb2-d4b3723146c3","Type":"ContainerDied","Data":"0119a0817e23cad126487b669d410d254a8db04d01c7bcceaabfdb2daaa82f1f"}
Sep 07 11:50:37.170104 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.170073    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:50:37.203305 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.203281    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:50:37.263933 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.263894    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1729c6be-0661-46e4-8bb2-d4b3723146c3-plugins-config-volume\") pod \"1729c6be-0661-46e4-8bb2-d4b3723146c3\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") "
Sep 07 11:50:37.264104 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.263981    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz7dv\" (UniqueName: \"kubernetes.io/projected/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-kube-api-access-nz7dv\") pod \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") "
Sep 07 11:50:37.264104 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.264070    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/1729c6be-0661-46e4-8bb2-d4b3723146c3-kube-api-access-cbgjb\") pod \"1729c6be-0661-46e4-8bb2-d4b3723146c3\" (UID: \"1729c6be-0661-46e4-8bb2-d4b3723146c3\") "
Sep 07 11:50:37.264196 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.264111    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-plugins-config-volume\") pod \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\" (UID: \"bb0dc8ae-c9b9-4975-910b-3150508bc1d5\") "
Sep 07 11:50:37.264482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.264338    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1729c6be-0661-46e4-8bb2-d4b3723146c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1729c6be-0661-46e4-8bb2-d4b3723146c3" (UID: "1729c6be-0661-46e4-8bb2-d4b3723146c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:50:37.264609 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.264537    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb0dc8ae-c9b9-4975-910b-3150508bc1d5" (UID: "bb0dc8ae-c9b9-4975-910b-3150508bc1d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:50:37.266220 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.266197    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1729c6be-0661-46e4-8bb2-d4b3723146c3-kube-api-access-cbgjb" (OuterVolumeSpecName: "kube-api-access-cbgjb") pod "1729c6be-0661-46e4-8bb2-d4b3723146c3" (UID: "1729c6be-0661-46e4-8bb2-d4b3723146c3"). InnerVolumeSpecName "kube-api-access-cbgjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:50:37.266659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.266628    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-kube-api-access-nz7dv" (OuterVolumeSpecName: "kube-api-access-nz7dv") pod "bb0dc8ae-c9b9-4975-910b-3150508bc1d5" (UID: "bb0dc8ae-c9b9-4975-910b-3150508bc1d5"). InnerVolumeSpecName "kube-api-access-nz7dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:50:37.365423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.365383    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1729c6be-0661-46e4-8bb2-d4b3723146c3-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:37.365423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.365420    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz7dv\" (UniqueName: \"kubernetes.io/projected/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-kube-api-access-nz7dv\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:37.365423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.365429    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/1729c6be-0661-46e4-8bb2-d4b3723146c3-kube-api-access-cbgjb\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:37.365423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:37.365438    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0dc8ae-c9b9-4975-910b-3150508bc1d5-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:50:38.130458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.130422    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"
Sep 07 11:50:38.130458 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.130443    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv" event={"ID":"bb0dc8ae-c9b9-4975-910b-3150508bc1d5","Type":"ContainerDied","Data":"31c5140d97f460952e8cb7ceacec065584f76c84226fe2e7aa325790587fadbe"}
Sep 07 11:50:38.131015 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.130514    2560 scope.go:117] "RemoveContainer" containerID="f3f99a11f69924e7e31a86e9621d87bde1ca3dce0b4b26fffa20b47b6f9961e0"
Sep 07 11:50:38.132326 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.132306    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t" event={"ID":"1729c6be-0661-46e4-8bb2-d4b3723146c3","Type":"ContainerDied","Data":"ae73a0157544294eb14c65c581692bab70319d449d93b551a67ce5f37983c25e"}
Sep 07 11:50:38.132407 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.132390    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"
Sep 07 11:50:38.152090 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.151958    2560 scope.go:117] "RemoveContainer" containerID="0119a0817e23cad126487b669d410d254a8db04d01c7bcceaabfdb2daaa82f1f"
Sep 07 11:50:38.166413 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.166384    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"]
Sep 07 11:50:38.172402 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.172377    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-984178-695974bbb-pzw8t"]
Sep 07 11:50:38.183453 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.183421    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"]
Sep 07 11:50:38.189209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.189169    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-984178-7964b67dcf-6zrfv"]
Sep 07 11:50:38.497645 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.497614    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1729c6be-0661-46e4-8bb2-d4b3723146c3" path="/var/lib/kubelet/pods/1729c6be-0661-46e4-8bb2-d4b3723146c3/volumes"
Sep 07 11:50:38.498029 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:50:38.498015    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb0dc8ae-c9b9-4975-910b-3150508bc1d5" path="/var/lib/kubelet/pods/bb0dc8ae-c9b9-4975-910b-3150508bc1d5/volumes"
Sep 07 11:51:26.162004 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:26.161966    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"]
Sep 07 11:51:26.162528 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:26.162230    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy" containerID="cri-o://77d64e7fcd408c376166e73edcf89cd1717c83db6dc7864681b9f2ffd7ce848c" gracePeriod=30
Sep 07 11:51:31.421147 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.421118    2560 generic.go:356] "Generic (PLEG): container finished" podID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerID="77d64e7fcd408c376166e73edcf89cd1717c83db6dc7864681b9f2ffd7ce848c" exitCode=0
Sep 07 11:51:31.421505 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.421191    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" event={"ID":"6a317fb2-b5ba-4016-b7d6-fc387531b848","Type":"ContainerDied","Data":"77d64e7fcd408c376166e73edcf89cd1717c83db6dc7864681b9f2ffd7ce848c"}
Sep 07 11:51:31.421505 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.421249    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z" event={"ID":"6a317fb2-b5ba-4016-b7d6-fc387531b848","Type":"ContainerDied","Data":"a0a9bdce4fa407c9a430fc4508c6e3348efef003baeba9ed64a96658ddc54f4a"}
Sep 07 11:51:31.421505 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.421267    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0a9bdce4fa407c9a430fc4508c6e3348efef003baeba9ed64a96658ddc54f4a"
Sep 07 11:51:31.421505 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.421281    2560 scope.go:117] "RemoveContainer" containerID="72576f518b3348a33b99621a8a0aa5a8edb1cbfe21738711304ed5ee980a3843"
Sep 07 11:51:31.422052 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.422034    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:51:31.593283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593249    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-podinfo\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593283 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593290    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a317fb2-b5ba-4016-b7d6-fc387531b848-istiod-ca-cert\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593316    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnp4v\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-kube-api-access-jnp4v\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593358    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-credential-socket\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593379    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-data\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593421    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-socket\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593522 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593495    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-envoy\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593809 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593596    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-token\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593809 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593648    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-certs\") pod \"6a317fb2-b5ba-4016-b7d6-fc387531b848\" (UID: \"6a317fb2-b5ba-4016-b7d6-fc387531b848\") "
Sep 07 11:51:31.593809 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593731    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a317fb2-b5ba-4016-b7d6-fc387531b848-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:51:31.593809 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:51:31.594029 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.593864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:51:31.594150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594104    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:51:31.594150 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594118    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:51:31.594275 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594255    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.594333 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594273    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a317fb2-b5ba-4016-b7d6-fc387531b848-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.594333 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594287    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.594333 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594301    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.594333 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.594315    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.595694 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.595669    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:51:31.595809 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.595786    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:51:31.595854 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.595823    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-kube-api-access-jnp4v" (OuterVolumeSpecName: "kube-api-access-jnp4v") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "kube-api-access-jnp4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:51:31.595926 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.595910    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a317fb2-b5ba-4016-b7d6-fc387531b848" (UID: "6a317fb2-b5ba-4016-b7d6-fc387531b848"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:51:31.695200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.695099    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.695200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.695148    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnp4v\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-kube-api-access-jnp4v\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.695200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.695159    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:31.695200 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:31.695166    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a317fb2-b5ba-4016-b7d6-fc387531b848-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:32.427188 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:32.427160    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"
Sep 07 11:51:32.458669 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:32.458638    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"]
Sep 07 11:51:32.462104 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:32.462079    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-984178-openshift-default-5d8dfcd9b8-htt2z"]
Sep 07 11:51:32.498412 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:32.498366    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" path="/var/lib/kubelet/pods/6a317fb2-b5ba-4016-b7d6-fc387531b848/volumes"
Sep 07 11:51:37.277650 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.277609    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"]
Sep 07 11:51:37.278173 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.277886    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k" podUID="d2f57016-b93c-4550-b03e-d07ce9b2c731" containerName="payload-processing" containerID="cri-o://db1724d1dd50c813922882013e1e627b854c9610a60f1e075b0eeec4b308eb4f" gracePeriod=30
Sep 07 11:51:37.454300 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.454268    2560 generic.go:356] "Generic (PLEG): container finished" podID="d2f57016-b93c-4550-b03e-d07ce9b2c731" containerID="db1724d1dd50c813922882013e1e627b854c9610a60f1e075b0eeec4b308eb4f" exitCode=0
Sep 07 11:51:37.454441 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.454347    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k" event={"ID":"d2f57016-b93c-4550-b03e-d07ce9b2c731","Type":"ContainerDied","Data":"db1724d1dd50c813922882013e1e627b854c9610a60f1e075b0eeec4b308eb4f"}
Sep 07 11:51:37.541472 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.541411    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:51:37.551184 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.551163    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djfs7\" (UniqueName: \"kubernetes.io/projected/d2f57016-b93c-4550-b03e-d07ce9b2c731-kube-api-access-djfs7\") pod \"d2f57016-b93c-4550-b03e-d07ce9b2c731\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") "
Sep 07 11:51:37.551303 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.551292    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f57016-b93c-4550-b03e-d07ce9b2c731-plugins-config-volume\") pod \"d2f57016-b93c-4550-b03e-d07ce9b2c731\" (UID: \"d2f57016-b93c-4550-b03e-d07ce9b2c731\") "
Sep 07 11:51:37.551684 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.551655    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2f57016-b93c-4550-b03e-d07ce9b2c731-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2f57016-b93c-4550-b03e-d07ce9b2c731" (UID: "d2f57016-b93c-4550-b03e-d07ce9b2c731"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:51:37.553121 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.553092    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2f57016-b93c-4550-b03e-d07ce9b2c731-kube-api-access-djfs7" (OuterVolumeSpecName: "kube-api-access-djfs7") pod "d2f57016-b93c-4550-b03e-d07ce9b2c731" (UID: "d2f57016-b93c-4550-b03e-d07ce9b2c731"). InnerVolumeSpecName "kube-api-access-djfs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:51:37.653000 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.652964    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f57016-b93c-4550-b03e-d07ce9b2c731-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:37.653000 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:37.652987    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djfs7\" (UniqueName: \"kubernetes.io/projected/d2f57016-b93c-4550-b03e-d07ce9b2c731-kube-api-access-djfs7\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:38.461209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:38.461180    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"
Sep 07 11:51:38.461209 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:38.461186    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k" event={"ID":"d2f57016-b93c-4550-b03e-d07ce9b2c731","Type":"ContainerDied","Data":"a991e78600e0c1603c14080b416d44f2657bc9913ea3053e439fc59eddcae7af"}
Sep 07 11:51:38.461679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:38.461241    2560 scope.go:117] "RemoveContainer" containerID="db1724d1dd50c813922882013e1e627b854c9610a60f1e075b0eeec4b308eb4f"
Sep 07 11:51:38.482822 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:38.482799    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"]
Sep 07 11:51:38.488619 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:38.488595    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f866d4-65bc4c7bc9-wnt5k"]
Sep 07 11:51:38.498009 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:38.497981    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2f57016-b93c-4550-b03e-d07ce9b2c731" path="/var/lib/kubelet/pods/d2f57016-b93c-4550-b03e-d07ce9b2c731/volumes"
Sep 07 11:51:44.438338 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438299    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"]
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438731    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8702ebe8-4b81-4161-8285-13b163c62943" containerName="payload-pre-processing"
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438757    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8702ebe8-4b81-4161-8285-13b163c62943" containerName="payload-pre-processing"
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438768    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0dc8ae-c9b9-4975-910b-3150508bc1d5" containerName="payload-processing"
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438774    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0dc8ae-c9b9-4975-910b-3150508bc1d5" containerName="payload-processing"
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438786    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17e66a86-b85c-4e30-b66a-b54d2395b07b" containerName="revoke-keys"
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438792    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="17e66a86-b85c-4e30-b66a-b54d2395b07b" containerName="revoke-keys"
Sep 07 11:51:44.438797 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438801    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438806    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438817    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438822    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438831    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1729c6be-0661-46e4-8bb2-d4b3723146c3" containerName="payload-pre-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438836    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1729c6be-0661-46e4-8bb2-d4b3723146c3" containerName="payload-pre-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438842    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f57016-b93c-4550-b03e-d07ce9b2c731" containerName="payload-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438847    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f57016-b93c-4550-b03e-d07ce9b2c731" containerName="payload-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438900    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1729c6be-0661-46e4-8bb2-d4b3723146c3" containerName="payload-pre-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438907    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2f57016-b93c-4550-b03e-d07ce9b2c731" containerName="payload-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438914    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8702ebe8-4b81-4161-8285-13b163c62943" containerName="payload-pre-processing"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438921    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438929    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a317fb2-b5ba-4016-b7d6-fc387531b848" containerName="istio-proxy"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438934    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="17e66a86-b85c-4e30-b66a-b54d2395b07b" containerName="revoke-keys"
Sep 07 11:51:44.439159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.438939    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0dc8ae-c9b9-4975-910b-3150508bc1d5" containerName="payload-processing"
Sep 07 11:51:44.443695 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.443676    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:44.445708 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.445690    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:51:44.449235 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.449209    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"]
Sep 07 11:51:44.514556 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.514520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22477e56-2e7f-4206-8c5e-cf27c1d7eed3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7\" (UID: \"22477e56-2e7f-4206-8c5e-cf27c1d7eed3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:44.616117 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.616081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22477e56-2e7f-4206-8c5e-cf27c1d7eed3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7\" (UID: \"22477e56-2e7f-4206-8c5e-cf27c1d7eed3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:44.616716 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.616697    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22477e56-2e7f-4206-8c5e-cf27c1d7eed3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7\" (UID: \"22477e56-2e7f-4206-8c5e-cf27c1d7eed3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:44.755656 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:44.755611    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:45.024216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:45.024189    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"]
Sep 07 11:51:45.026382 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:51:45.026352    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22477e56_2e7f_4206_8c5e_cf27c1d7eed3.slice/crio-cd120f7572eee2ff24ba37772511c3e7114b1e143909eab71e0c38f0af85694c WatchSource:0}: Error finding container cd120f7572eee2ff24ba37772511c3e7114b1e143909eab71e0c38f0af85694c: Status 404 returned error can't find the container with id cd120f7572eee2ff24ba37772511c3e7114b1e143909eab71e0c38f0af85694c
Sep 07 11:51:45.502839 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:45.502805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7" event={"ID":"22477e56-2e7f-4206-8c5e-cf27c1d7eed3","Type":"ContainerStarted","Data":"d135487ed7f2035a109fbdf0553a3bd6ade0849b72d3d9a61ddb82952a46159d"}
Sep 07 11:51:45.503231 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:45.502854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7" event={"ID":"22477e56-2e7f-4206-8c5e-cf27c1d7eed3","Type":"ContainerStarted","Data":"cd120f7572eee2ff24ba37772511c3e7114b1e143909eab71e0c38f0af85694c"}
Sep 07 11:51:45.518118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:45.518069    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7" podStartSLOduration=1.518054759 podStartE2EDuration="1.518054759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:45.516575182 +0000 UTC m=+1737.463925966" watchObservedRunningTime="2026-09-07 11:51:45.518054759 +0000 UTC m=+1737.465405580"
Sep 07 11:51:46.510191 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:46.510159    2560 generic.go:356] "Generic (PLEG): container finished" podID="22477e56-2e7f-4206-8c5e-cf27c1d7eed3" containerID="d135487ed7f2035a109fbdf0553a3bd6ade0849b72d3d9a61ddb82952a46159d" exitCode=0
Sep 07 11:51:46.510659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:46.510239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7" event={"ID":"22477e56-2e7f-4206-8c5e-cf27c1d7eed3","Type":"ContainerDied","Data":"d135487ed7f2035a109fbdf0553a3bd6ade0849b72d3d9a61ddb82952a46159d"}
Sep 07 11:51:47.648861 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:47.648832    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:47.746091 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:47.746055    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22477e56-2e7f-4206-8c5e-cf27c1d7eed3-maas-api-service-ca\") pod \"22477e56-2e7f-4206-8c5e-cf27c1d7eed3\" (UID: \"22477e56-2e7f-4206-8c5e-cf27c1d7eed3\") "
Sep 07 11:51:47.746362 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:47.746338    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22477e56-2e7f-4206-8c5e-cf27c1d7eed3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22477e56-2e7f-4206-8c5e-cf27c1d7eed3" (UID: "22477e56-2e7f-4206-8c5e-cf27c1d7eed3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:51:47.847558 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:47.847469    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22477e56-2e7f-4206-8c5e-cf27c1d7eed3-maas-api-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:48.521951 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:48.521925    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"
Sep 07 11:51:48.522096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:48.521958    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7" event={"ID":"22477e56-2e7f-4206-8c5e-cf27c1d7eed3","Type":"ContainerDied","Data":"cd120f7572eee2ff24ba37772511c3e7114b1e143909eab71e0c38f0af85694c"}
Sep 07 11:51:48.522096 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:48.521997    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd120f7572eee2ff24ba37772511c3e7114b1e143909eab71e0c38f0af85694c"
Sep 07 11:51:48.825351 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:48.825264    2560 scope.go:117] "RemoveContainer" containerID="72722c089c30cfab4fcec517f689f2cf707adf388a51411bd04c64d9d366672d"
Sep 07 11:51:55.146725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.146685    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"]
Sep 07 11:51:55.147230 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.146933    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh" podUID="ed8c689f-1d27-4f58-8466-f6363bde92ac" containerName="payload-processing" containerID="cri-o://9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae" gracePeriod=30
Sep 07 11:51:55.167914 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.167886    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"]
Sep 07 11:51:55.168144 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.168104    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5" podUID="291646c7-4f9c-43ac-be2a-ca9ed15a37d1" containerName="payload-pre-processing" containerID="cri-o://84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd" gracePeriod=30
Sep 07 11:51:55.383314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.383281    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"]
Sep 07 11:51:55.385847 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.385820    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6b944b-6wjk7"]
Sep 07 11:51:55.435302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.435280    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:51:55.441099 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.441079    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:51:55.521527 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.521494    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8c689f-1d27-4f58-8466-f6363bde92ac-plugins-config-volume\") pod \"ed8c689f-1d27-4f58-8466-f6363bde92ac\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") "
Sep 07 11:51:55.521681 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.521573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-plugins-config-volume\") pod \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") "
Sep 07 11:51:55.521785 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.521734    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fqwk\" (UniqueName: \"kubernetes.io/projected/ed8c689f-1d27-4f58-8466-f6363bde92ac-kube-api-access-4fqwk\") pod \"ed8c689f-1d27-4f58-8466-f6363bde92ac\" (UID: \"ed8c689f-1d27-4f58-8466-f6363bde92ac\") "
Sep 07 11:51:55.521895 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.521877    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdrcc\" (UniqueName: \"kubernetes.io/projected/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-kube-api-access-mdrcc\") pod \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\" (UID: \"291646c7-4f9c-43ac-be2a-ca9ed15a37d1\") "
Sep 07 11:51:55.521966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.521893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed8c689f-1d27-4f58-8466-f6363bde92ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed8c689f-1d27-4f58-8466-f6363bde92ac" (UID: "ed8c689f-1d27-4f58-8466-f6363bde92ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:51:55.521966 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.521952    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "291646c7-4f9c-43ac-be2a-ca9ed15a37d1" (UID: "291646c7-4f9c-43ac-be2a-ca9ed15a37d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:51:55.522170 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.522155    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8c689f-1d27-4f58-8466-f6363bde92ac-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:55.522210 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.522171    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:55.523764 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.523708    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed8c689f-1d27-4f58-8466-f6363bde92ac-kube-api-access-4fqwk" (OuterVolumeSpecName: "kube-api-access-4fqwk") pod "ed8c689f-1d27-4f58-8466-f6363bde92ac" (UID: "ed8c689f-1d27-4f58-8466-f6363bde92ac"). InnerVolumeSpecName "kube-api-access-4fqwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:51:55.523880 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.523835    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-kube-api-access-mdrcc" (OuterVolumeSpecName: "kube-api-access-mdrcc") pod "291646c7-4f9c-43ac-be2a-ca9ed15a37d1" (UID: "291646c7-4f9c-43ac-be2a-ca9ed15a37d1"). InnerVolumeSpecName "kube-api-access-mdrcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:51:55.556603 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.556572    2560 generic.go:356] "Generic (PLEG): container finished" podID="ed8c689f-1d27-4f58-8466-f6363bde92ac" containerID="9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae" exitCode=0
Sep 07 11:51:55.556712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.556623    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"
Sep 07 11:51:55.556712 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.556617    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh" event={"ID":"ed8c689f-1d27-4f58-8466-f6363bde92ac","Type":"ContainerDied","Data":"9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae"}
Sep 07 11:51:55.556846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.556753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh" event={"ID":"ed8c689f-1d27-4f58-8466-f6363bde92ac","Type":"ContainerDied","Data":"f3fb574f86351c8e3d1dd21a18df12200aa50079b5de237604e2124e6d6956f9"}
Sep 07 11:51:55.556846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.556783    2560 scope.go:117] "RemoveContainer" containerID="9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae"
Sep 07 11:51:55.558159 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.558137    2560 generic.go:356] "Generic (PLEG): container finished" podID="291646c7-4f9c-43ac-be2a-ca9ed15a37d1" containerID="84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd" exitCode=0
Sep 07 11:51:55.558286 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.558164    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5" event={"ID":"291646c7-4f9c-43ac-be2a-ca9ed15a37d1","Type":"ContainerDied","Data":"84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd"}
Sep 07 11:51:55.558286 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.558194    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"
Sep 07 11:51:55.558286 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.558216    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5" event={"ID":"291646c7-4f9c-43ac-be2a-ca9ed15a37d1","Type":"ContainerDied","Data":"39cbb1539e0677541a161c522c145ca167cd4563c6c85fa3eddc6105e538a03c"}
Sep 07 11:51:55.566482 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.566467    2560 scope.go:117] "RemoveContainer" containerID="9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae"
Sep 07 11:51:55.566989 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:51:55.566880    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae\": container with ID starting with 9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae not found: ID does not exist" containerID="9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae"
Sep 07 11:51:55.566989 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.566919    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae"} err="failed to get container status \"9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae\": rpc error: code = NotFound desc = could not find container \"9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae\": container with ID starting with 9479882e67c58d6a2d83892f9059a3ce5a21213c0348ec8e33fad9a15e8d6eae not found: ID does not exist"
Sep 07 11:51:55.566989 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.566943    2560 scope.go:117] "RemoveContainer" containerID="84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd"
Sep 07 11:51:55.574899 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.574880    2560 scope.go:117] "RemoveContainer" containerID="84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd"
Sep 07 11:51:55.575179 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:51:55.575159    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd\": container with ID starting with 84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd not found: ID does not exist" containerID="84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd"
Sep 07 11:51:55.575242 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.575191    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd"} err="failed to get container status \"84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd\": rpc error: code = NotFound desc = could not find container \"84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd\": container with ID starting with 84f323035907bd9c78bdd8b1988564f74719931e2441d033e966ebcce3c784cd not found: ID does not exist"
Sep 07 11:51:55.587568 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.587542    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"]
Sep 07 11:51:55.592997 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.592978    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6b944b-64c8cdd84b-2zjm5"]
Sep 07 11:51:55.608575 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.608554    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"]
Sep 07 11:51:55.613399 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.613382    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6b944b-759cf49565-xgjdh"]
Sep 07 11:51:55.622922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.622903    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fqwk\" (UniqueName: \"kubernetes.io/projected/ed8c689f-1d27-4f58-8466-f6363bde92ac-kube-api-access-4fqwk\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:55.622979 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:55.622925    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdrcc\" (UniqueName: \"kubernetes.io/projected/291646c7-4f9c-43ac-be2a-ca9ed15a37d1-kube-api-access-mdrcc\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:51:56.504281 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:56.504245    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22477e56-2e7f-4206-8c5e-cf27c1d7eed3" path="/var/lib/kubelet/pods/22477e56-2e7f-4206-8c5e-cf27c1d7eed3/volumes"
Sep 07 11:51:56.504725 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:56.504613    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291646c7-4f9c-43ac-be2a-ca9ed15a37d1" path="/var/lib/kubelet/pods/291646c7-4f9c-43ac-be2a-ca9ed15a37d1/volumes"
Sep 07 11:51:56.504997 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:51:56.504982    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8c689f-1d27-4f58-8466-f6363bde92ac" path="/var/lib/kubelet/pods/ed8c689f-1d27-4f58-8466-f6363bde92ac/volumes"
Sep 07 11:52:02.333454 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.333417    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"]
Sep 07 11:52:02.333837 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.333710    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy" containerID="cri-o://f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c" gracePeriod=30
Sep 07 11:52:02.675574 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675492    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"]
Sep 07 11:52:02.675922 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675908    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22477e56-2e7f-4206-8c5e-cf27c1d7eed3" containerName="revoke-keys"
Sep 07 11:52:02.675975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675924    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="22477e56-2e7f-4206-8c5e-cf27c1d7eed3" containerName="revoke-keys"
Sep 07 11:52:02.675975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675944    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291646c7-4f9c-43ac-be2a-ca9ed15a37d1" containerName="payload-pre-processing"
Sep 07 11:52:02.675975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675950    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="291646c7-4f9c-43ac-be2a-ca9ed15a37d1" containerName="payload-pre-processing"
Sep 07 11:52:02.675975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675964    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8c689f-1d27-4f58-8466-f6363bde92ac" containerName="payload-processing"
Sep 07 11:52:02.675975 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.675969    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8c689f-1d27-4f58-8466-f6363bde92ac" containerName="payload-processing"
Sep 07 11:52:02.676132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.676036    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed8c689f-1d27-4f58-8466-f6363bde92ac" containerName="payload-processing"
Sep 07 11:52:02.676132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.676044    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="22477e56-2e7f-4206-8c5e-cf27c1d7eed3" containerName="revoke-keys"
Sep 07 11:52:02.676132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.676052    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="291646c7-4f9c-43ac-be2a-ca9ed15a37d1" containerName="payload-pre-processing"
Sep 07 11:52:02.680624 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.680609    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:02.682806 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.682783    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5cm8r\""
Sep 07 11:52:02.688716 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.688289    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"]
Sep 07 11:52:02.791960 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.791921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz\" (UID: \"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:02.893592 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.893547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz\" (UID: \"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:02.894253 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:02.894203    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz\" (UID: \"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:03.002227 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:03.002194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:03.273198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:03.273090    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"]
Sep 07 11:52:03.275538 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:52:03.275498    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e4f6cd_9e02_48ac_b855_f1aeaa1e6dd7.slice/crio-ded89ee80f28af54db87bedf61ec49e1f342204badd3d8753b83e815f869e4e5 WatchSource:0}: Error finding container ded89ee80f28af54db87bedf61ec49e1f342204badd3d8753b83e815f869e4e5: Status 404 returned error can't find the container with id ded89ee80f28af54db87bedf61ec49e1f342204badd3d8753b83e815f869e4e5
Sep 07 11:52:03.603569 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:03.603531    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz" event={"ID":"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7","Type":"ContainerStarted","Data":"ded89ee80f28af54db87bedf61ec49e1f342204badd3d8753b83e815f869e4e5"}
Sep 07 11:52:04.609530 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:04.609494    2560 generic.go:356] "Generic (PLEG): container finished" podID="f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7" containerID="31366058b2c401d9af6e8b82a9faf7cbeea7e3113e9d4658c0d44e0dfccbe20e" exitCode=0
Sep 07 11:52:04.609927 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:04.609545    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz" event={"ID":"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7","Type":"ContainerDied","Data":"31366058b2c401d9af6e8b82a9faf7cbeea7e3113e9d4658c0d44e0dfccbe20e"}
Sep 07 11:52:05.746654 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:05.746633    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:05.924233 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:05.924149    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7-maas-api-service-ca\") pod \"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7\" (UID: \"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7\") "
Sep 07 11:52:05.924487 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:05.924465    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7" (UID: "f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:52:06.025118 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:06.025066    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7-maas-api-service-ca\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:06.621411 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:06.621377    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"
Sep 07 11:52:06.621411 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:06.621386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz" event={"ID":"f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7","Type":"ContainerDied","Data":"ded89ee80f28af54db87bedf61ec49e1f342204badd3d8753b83e815f869e4e5"}
Sep 07 11:52:06.621617 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:06.621439    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ded89ee80f28af54db87bedf61ec49e1f342204badd3d8753b83e815f869e4e5"
Sep 07 11:52:07.598987 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.598922    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:52:07.630103 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.630070    2560 generic.go:356] "Generic (PLEG): container finished" podID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerID="f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c" exitCode=0
Sep 07 11:52:07.630255 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.630107    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"
Sep 07 11:52:07.630255 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.630153    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" event={"ID":"13fd4eb3-74a9-448b-83bc-56c180f6355d","Type":"ContainerDied","Data":"f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c"}
Sep 07 11:52:07.630255 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.630214    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2" event={"ID":"13fd4eb3-74a9-448b-83bc-56c180f6355d","Type":"ContainerDied","Data":"887a3fc0cd76b85da5269c8777a3becb2eb579070a8781a6c05d476b072ebdef"}
Sep 07 11:52:07.630255 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.630248    2560 scope.go:117] "RemoveContainer" containerID="f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c"
Sep 07 11:52:07.640943 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.640926    2560 scope.go:117] "RemoveContainer" containerID="1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"
Sep 07 11:52:07.648794 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.648773    2560 scope.go:117] "RemoveContainer" containerID="f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c"
Sep 07 11:52:07.649078 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:52:07.649057    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c\": container with ID starting with f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c not found: ID does not exist" containerID="f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c"
Sep 07 11:52:07.649158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.649084    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c"} err="failed to get container status \"f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c\": rpc error: code = NotFound desc = could not find container \"f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c\": container with ID starting with f19563393ed5fdb0817cec66cb0418e8bfcda9f07a2d3865c31e555e915c1f9c not found: ID does not exist"
Sep 07 11:52:07.649158 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.649101    2560 scope.go:117] "RemoveContainer" containerID="1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"
Sep 07 11:52:07.649400 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:52:07.649382    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b\": container with ID starting with 1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b not found: ID does not exist" containerID="1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"
Sep 07 11:52:07.649469 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.649406    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b"} err="failed to get container status \"1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b\": rpc error: code = NotFound desc = could not find container \"1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b\": container with ID starting with 1975cc9733b4bd6c6e0a0b351fa379c1227e69729d1f016a27b48d8bd768889b not found: ID does not exist"
Sep 07 11:52:07.742422 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742394    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-envoy\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742435    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-podinfo\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742465    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-credential-socket\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742551 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742485    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-socket\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742566    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13fd4eb3-74a9-448b-83bc-56c180f6355d-istiod-ca-cert\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742590    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhbp6\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-kube-api-access-vhbp6\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742637    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-data\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742722 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742654    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-certs\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742958 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742813    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:52:07.742958 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742869    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-token\") pod \"13fd4eb3-74a9-448b-83bc-56c180f6355d\" (UID: \"13fd4eb3-74a9-448b-83bc-56c180f6355d\") "
Sep 07 11:52:07.742958 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.742934    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-data" (OuterVolumeSpecName: "istio-data") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:52:07.743109 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.743036    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:52:07.743261 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.743151    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13fd4eb3-74a9-448b-83bc-56c180f6355d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:52:07.743314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.743281    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:52:07.743314 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.743287    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-data\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.743378 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.743327    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-credential-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.743378 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.743343    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-socket\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.744731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.744708    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-kube-api-access-vhbp6" (OuterVolumeSpecName: "kube-api-access-vhbp6") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "kube-api-access-vhbp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:52:07.744977 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.744957    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:52:07.744977 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.744968    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-token" (OuterVolumeSpecName: "istio-token") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:52:07.745124 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.745107    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13fd4eb3-74a9-448b-83bc-56c180f6355d" (UID: "13fd4eb3-74a9-448b-83bc-56c180f6355d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:52:07.844182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.844153    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13fd4eb3-74a9-448b-83bc-56c180f6355d-istiod-ca-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.844182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.844173    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhbp6\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-kube-api-access-vhbp6\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.844182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.844183    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-workload-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.844341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.844193    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-token\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.844341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.844201    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-envoy\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.844341 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.844208    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13fd4eb3-74a9-448b-83bc-56c180f6355d-istio-podinfo\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:07.953110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.953077    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"]
Sep 07 11:52:07.958849 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:07.958828    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b944b-openshift-default-7cb776f4fd-477f2"]
Sep 07 11:52:08.504659 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:08.504627    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" path="/var/lib/kubelet/pods/13fd4eb3-74a9-448b-83bc-56c180f6355d/volumes"
Sep 07 11:52:13.146491 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.146457    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"]
Sep 07 11:52:13.148197 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.148176    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0f0a2f-29813025-48nhx"]
Sep 07 11:52:13.179588 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.179558    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"]
Sep 07 11:52:13.179829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.179801    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj" podUID="628ef7d0-f696-4773-9a78-670ee59d2cbd" containerName="payload-pre-processing" containerID="cri-o://186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2" gracePeriod=30
Sep 07 11:52:13.329146 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.329088    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"]
Sep 07 11:52:13.331001 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.330978    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0f0a2f-msqdz"]
Sep 07 11:52:13.427945 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.427915    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:52:13.491812 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.491783    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628ef7d0-f696-4773-9a78-670ee59d2cbd-plugins-config-volume\") pod \"628ef7d0-f696-4773-9a78-670ee59d2cbd\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") "
Sep 07 11:52:13.491978 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.491830    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gr5q\" (UniqueName: \"kubernetes.io/projected/628ef7d0-f696-4773-9a78-670ee59d2cbd-kube-api-access-7gr5q\") pod \"628ef7d0-f696-4773-9a78-670ee59d2cbd\" (UID: \"628ef7d0-f696-4773-9a78-670ee59d2cbd\") "
Sep 07 11:52:13.492198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.492177    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628ef7d0-f696-4773-9a78-670ee59d2cbd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "628ef7d0-f696-4773-9a78-670ee59d2cbd" (UID: "628ef7d0-f696-4773-9a78-670ee59d2cbd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:52:13.493846 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.493823    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628ef7d0-f696-4773-9a78-670ee59d2cbd-kube-api-access-7gr5q" (OuterVolumeSpecName: "kube-api-access-7gr5q") pod "628ef7d0-f696-4773-9a78-670ee59d2cbd" (UID: "628ef7d0-f696-4773-9a78-670ee59d2cbd"). InnerVolumeSpecName "kube-api-access-7gr5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:52:13.593122 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.593088    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/628ef7d0-f696-4773-9a78-670ee59d2cbd-plugins-config-volume\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:13.593122 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.593115    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gr5q\" (UniqueName: \"kubernetes.io/projected/628ef7d0-f696-4773-9a78-670ee59d2cbd-kube-api-access-7gr5q\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:52:13.662552 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.662470    2560 generic.go:356] "Generic (PLEG): container finished" podID="628ef7d0-f696-4773-9a78-670ee59d2cbd" containerID="186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2" exitCode=0
Sep 07 11:52:13.662552 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.662522    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"
Sep 07 11:52:13.662724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.662567    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj" event={"ID":"628ef7d0-f696-4773-9a78-670ee59d2cbd","Type":"ContainerDied","Data":"186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2"}
Sep 07 11:52:13.662724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.662615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj" event={"ID":"628ef7d0-f696-4773-9a78-670ee59d2cbd","Type":"ContainerDied","Data":"daa8b339b185e89fbc0f71cca54a3f80fefd3947980c1a8dbbc3499659364783"}
Sep 07 11:52:13.662724 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.662643    2560 scope.go:117] "RemoveContainer" containerID="186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2"
Sep 07 11:52:13.677058 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.677033    2560 scope.go:117] "RemoveContainer" containerID="186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2"
Sep 07 11:52:13.677326 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:52:13.677307    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2\": container with ID starting with 186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2 not found: ID does not exist" containerID="186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2"
Sep 07 11:52:13.677377 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.677333    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2"} err="failed to get container status \"186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2\": rpc error: code = NotFound desc = could not find container \"186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2\": container with ID starting with 186691e3c3700ef0aba8ac8ac0b20ad62a97455e1d96515939668e78fb7d9ef2 not found: ID does not exist"
Sep 07 11:52:13.688965 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.688938    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"]
Sep 07 11:52:13.692609 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:13.692590    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0f0a2f-57986bbdf8-sxnfj"]
Sep 07 11:52:14.497055 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:14.497021    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d7b65e8-d5c9-41f0-9571-0230b299ddb7" path="/var/lib/kubelet/pods/2d7b65e8-d5c9-41f0-9571-0230b299ddb7/volumes"
Sep 07 11:52:14.497423 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:14.497340    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628ef7d0-f696-4773-9a78-670ee59d2cbd" path="/var/lib/kubelet/pods/628ef7d0-f696-4773-9a78-670ee59d2cbd/volumes"
Sep 07 11:52:14.497618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:14.497604    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7" path="/var/lib/kubelet/pods/f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7/volumes"
Sep 07 11:52:48.635155 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:48.635126    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:52:48.640431 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:48.640409    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:52:48.848717 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:52:48.848688    2560 scope.go:117] "RemoveContainer" containerID="d499c79b5a7a2ed2a13a17199972646082288a5e61e2d02ba4a988fe171abe4f"
Sep 07 11:53:25.248648 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.248613    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-xfpx8"]
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249007    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7" containerName="revoke-keys"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249021    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7" containerName="revoke-keys"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249042    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628ef7d0-f696-4773-9a78-670ee59d2cbd" containerName="payload-pre-processing"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249049    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="628ef7d0-f696-4773-9a78-670ee59d2cbd" containerName="payload-pre-processing"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249059    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249065    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249078    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy"
Sep 07 11:53:25.249138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249083    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy"
Sep 07 11:53:25.249393 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249144    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy"
Sep 07 11:53:25.249393 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249152    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="628ef7d0-f696-4773-9a78-670ee59d2cbd" containerName="payload-pre-processing"
Sep 07 11:53:25.249393 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.249159    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e4f6cd-9e02-48ac-b855-f1aeaa1e6dd7" containerName="revoke-keys"
Sep 07 11:53:25.252330 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.252311    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.254935 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.254918    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 11:53:25.254935 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.254927    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 11:53:25.255106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.255044    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 11:53:25.255106 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.255082    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4pz7p\""
Sep 07 11:53:25.262923 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.262901    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-xfpx8"]
Sep 07 11:53:25.351390 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.351351    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-webhook-cert\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.351390 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.351392    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26tgw\" (UniqueName: \"kubernetes.io/projected/3c1ee314-54a1-4586-8717-1f07f92139ae-kube-api-access-26tgw\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.351652 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.351483    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-metrics-certs\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.452618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.452572    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-webhook-cert\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.452618 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.452622    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26tgw\" (UniqueName: \"kubernetes.io/projected/3c1ee314-54a1-4586-8717-1f07f92139ae-kube-api-access-26tgw\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.452843 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.452672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-metrics-certs\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.455242 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.455216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-metrics-certs\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.455347 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.455216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-webhook-cert\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.460441 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.460417    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26tgw\" (UniqueName: \"kubernetes.io/projected/3c1ee314-54a1-4586-8717-1f07f92139ae-kube-api-access-26tgw\") pod \"maas-controller-f74dc8dd5-xfpx8\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") " pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.563344 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.563195    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:25.831955 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.831930    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-xfpx8"]
Sep 07 11:53:25.835118 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:53:25.835089    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1ee314_54a1_4586_8717_1f07f92139ae.slice/crio-ff104ea5f1bc4491c2d49a508c5c0f76726a273feb26c23231be22a692d21f20 WatchSource:0}: Error finding container ff104ea5f1bc4491c2d49a508c5c0f76726a273feb26c23231be22a692d21f20: Status 404 returned error can't find the container with id ff104ea5f1bc4491c2d49a508c5c0f76726a273feb26c23231be22a692d21f20
Sep 07 11:53:25.839471 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:25.838279    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:53:26.030429 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:26.030394    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8" event={"ID":"3c1ee314-54a1-4586-8717-1f07f92139ae","Type":"ContainerStarted","Data":"ff104ea5f1bc4491c2d49a508c5c0f76726a273feb26c23231be22a692d21f20"}
Sep 07 11:53:27.038073 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:27.038037    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8" event={"ID":"3c1ee314-54a1-4586-8717-1f07f92139ae","Type":"ContainerStarted","Data":"f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd"}
Sep 07 11:53:27.038455 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:27.038135    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:27.055332 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:27.055277    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8" podStartSLOduration=1.616828343 podStartE2EDuration="2.055261793s" totalImagesPullingTime="438.43345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:27.053358959 +0000 UTC m=+1839.000709743" watchObservedRunningTime="2026-09-07 11:53:27.055261793 +0000 UTC m=+1839.002612576"
Sep 07 11:53:38.045731 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:38.045696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:47.576557 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.576519    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-xfpx8"]
Sep 07 11:53:47.577066 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.576755    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8" podUID="3c1ee314-54a1-4586-8717-1f07f92139ae" containerName="manager" containerID="cri-o://f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd" gracePeriod=30
Sep 07 11:53:47.837678 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.837611    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:47.952110 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.952062    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-metrics-certs\") pod \"3c1ee314-54a1-4586-8717-1f07f92139ae\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") "
Sep 07 11:53:47.952285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.952153    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26tgw\" (UniqueName: \"kubernetes.io/projected/3c1ee314-54a1-4586-8717-1f07f92139ae-kube-api-access-26tgw\") pod \"3c1ee314-54a1-4586-8717-1f07f92139ae\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") "
Sep 07 11:53:47.952285 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.952258    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-webhook-cert\") pod \"3c1ee314-54a1-4586-8717-1f07f92139ae\" (UID: \"3c1ee314-54a1-4586-8717-1f07f92139ae\") "
Sep 07 11:53:47.954497 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.954466    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1ee314-54a1-4586-8717-1f07f92139ae-kube-api-access-26tgw" (OuterVolumeSpecName: "kube-api-access-26tgw") pod "3c1ee314-54a1-4586-8717-1f07f92139ae" (UID: "3c1ee314-54a1-4586-8717-1f07f92139ae"). InnerVolumeSpecName "kube-api-access-26tgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:53:47.954905 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.954885    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c1ee314-54a1-4586-8717-1f07f92139ae" (UID: "3c1ee314-54a1-4586-8717-1f07f92139ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:47.955070 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:47.955036    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3c1ee314-54a1-4586-8717-1f07f92139ae" (UID: "3c1ee314-54a1-4586-8717-1f07f92139ae"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:48.053460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.053404    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-metrics-certs\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:53:48.053460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.053450    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26tgw\" (UniqueName: \"kubernetes.io/projected/3c1ee314-54a1-4586-8717-1f07f92139ae-kube-api-access-26tgw\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:53:48.053460 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.053466    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c1ee314-54a1-4586-8717-1f07f92139ae-webhook-cert\") on node \"ip-10-0-138-45.ec2.internal\" DevicePath \"\""
Sep 07 11:53:48.154593 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.154501    2560 generic.go:356] "Generic (PLEG): container finished" podID="3c1ee314-54a1-4586-8717-1f07f92139ae" containerID="f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd" exitCode=0
Sep 07 11:53:48.154593 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.154550    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8"
Sep 07 11:53:48.154831 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.154553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8" event={"ID":"3c1ee314-54a1-4586-8717-1f07f92139ae","Type":"ContainerDied","Data":"f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd"}
Sep 07 11:53:48.154831 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.154680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f74dc8dd5-xfpx8" event={"ID":"3c1ee314-54a1-4586-8717-1f07f92139ae","Type":"ContainerDied","Data":"ff104ea5f1bc4491c2d49a508c5c0f76726a273feb26c23231be22a692d21f20"}
Sep 07 11:53:48.154831 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.154708    2560 scope.go:117] "RemoveContainer" containerID="f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd"
Sep 07 11:53:48.164281 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.164262    2560 scope.go:117] "RemoveContainer" containerID="f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd"
Sep 07 11:53:48.164574 ip-10-0-138-45 kubenswrapper[2560]: E0907 11:53:48.164554    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd\": container with ID starting with f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd not found: ID does not exist" containerID="f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd"
Sep 07 11:53:48.164636 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.164582    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd"} err="failed to get container status \"f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd\": rpc error: code = NotFound desc = could not find container \"f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd\": container with ID starting with f0b8270b123835f7c3741e6b1a0bbac92028ebad27db2c65ef7ffe4d5f4025bd not found: ID does not exist"
Sep 07 11:53:48.176055 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.176032    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-xfpx8"]
Sep 07 11:53:48.179696 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.179665    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-xfpx8"]
Sep 07 11:53:48.497141 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:48.497106    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c1ee314-54a1-4586-8717-1f07f92139ae" path="/var/lib/kubelet/pods/3c1ee314-54a1-4586-8717-1f07f92139ae/volumes"
Sep 07 11:53:49.068924 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.068891    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-h6hdc"]
Sep 07 11:53:49.069292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.069239    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c1ee314-54a1-4586-8717-1f07f92139ae" containerName="manager"
Sep 07 11:53:49.069292 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.069250    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c1ee314-54a1-4586-8717-1f07f92139ae" containerName="manager"
Sep 07 11:53:49.069364 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.069321    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c1ee314-54a1-4586-8717-1f07f92139ae" containerName="manager"
Sep 07 11:53:49.069364 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.069332    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13fd4eb3-74a9-448b-83bc-56c180f6355d" containerName="istio-proxy"
Sep 07 11:53:49.073650 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.073633    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.076518 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.076490    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 11:53:49.076646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.076563    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4pz7p\""
Sep 07 11:53:49.076646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.076491    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 11:53:49.076646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.076491    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 11:53:49.081457 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.081436    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-h6hdc"]
Sep 07 11:53:49.165366 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.165278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fad53d-2aa4-42a6-99bf-da92e7a425f0-webhook-cert\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.165366 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.165338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61fad53d-2aa4-42a6-99bf-da92e7a425f0-metrics-certs\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.165582 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.165418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8565\" (UniqueName: \"kubernetes.io/projected/61fad53d-2aa4-42a6-99bf-da92e7a425f0-kube-api-access-g8565\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.266611 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.266569    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8565\" (UniqueName: \"kubernetes.io/projected/61fad53d-2aa4-42a6-99bf-da92e7a425f0-kube-api-access-g8565\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.266842 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.266662    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fad53d-2aa4-42a6-99bf-da92e7a425f0-webhook-cert\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.266842 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.266726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61fad53d-2aa4-42a6-99bf-da92e7a425f0-metrics-certs\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.269298 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.269270    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61fad53d-2aa4-42a6-99bf-da92e7a425f0-metrics-certs\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.269398 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.269270    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fad53d-2aa4-42a6-99bf-da92e7a425f0-webhook-cert\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.275689 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.275664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8565\" (UniqueName: \"kubernetes.io/projected/61fad53d-2aa4-42a6-99bf-da92e7a425f0-kube-api-access-g8565\") pod \"maas-controller-f74dc8dd5-h6hdc\" (UID: \"61fad53d-2aa4-42a6-99bf-da92e7a425f0\") " pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.385069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.385031    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:49.657208 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:49.657182    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f74dc8dd5-h6hdc"]
Sep 07 11:53:49.659023 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:53:49.658994    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fad53d_2aa4_42a6_99bf_da92e7a425f0.slice/crio-263f0005497348073888ae38026ece273f7d04c8a11035e15a9e8fc80d976c0b WatchSource:0}: Error finding container 263f0005497348073888ae38026ece273f7d04c8a11035e15a9e8fc80d976c0b: Status 404 returned error can't find the container with id 263f0005497348073888ae38026ece273f7d04c8a11035e15a9e8fc80d976c0b
Sep 07 11:53:50.165586 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:50.165546    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc" event={"ID":"61fad53d-2aa4-42a6-99bf-da92e7a425f0","Type":"ContainerStarted","Data":"c84f2cae1a4afd99ae23e10d0b3be06817b2369f6033ff6aad77212257921a3a"}
Sep 07 11:53:50.165980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:50.165602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc" event={"ID":"61fad53d-2aa4-42a6-99bf-da92e7a425f0","Type":"ContainerStarted","Data":"263f0005497348073888ae38026ece273f7d04c8a11035e15a9e8fc80d976c0b"}
Sep 07 11:53:50.165980 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:50.165694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:53:50.183616 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:53:50.183569    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc" podStartSLOduration=0.803085629 podStartE2EDuration="1.18355502s" totalImagesPullingTime="380.469391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:50.181873552 +0000 UTC m=+1862.129224336" watchObservedRunningTime="2026-09-07 11:53:50.18355502 +0000 UTC m=+1862.130905804"
Sep 07 11:54:01.172359 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:54:01.172329    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f74dc8dd5-h6hdc"
Sep 07 11:54:48.894883 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:54:48.894848    2560 scope.go:117] "RemoveContainer" containerID="77d64e7fcd408c376166e73edcf89cd1717c83db6dc7864681b9f2ffd7ce848c"
Sep 07 11:56:48.923400 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:56:48.923371    2560 scope.go:117] "RemoveContainer" containerID="97ae3c8d8b52b08d8327cd02bd426f226d621114234ad7239ec830b829b4a597"
Sep 07 11:57:48.666444 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:57:48.666417    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:57:48.671100 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:57:48.671079    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8clx_a05961a1-9a71-424f-b728-7764722ae488/ovn-acl-logging/0.log"
Sep 07 11:57:48.944775 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:57:48.944681    2560 scope.go:117] "RemoveContainer" containerID="d135487ed7f2035a109fbdf0553a3bd6ade0849b72d3d9a61ddb82952a46159d"
Sep 07 11:58:14.020557 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.020507    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"]
Sep 07 11:58:14.024147 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.024126    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.026674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.026642    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gtjll\""
Sep 07 11:58:14.026674 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.026642    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 11:58:14.026909 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.026653    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 11:58:14.035459 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.035431    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"]
Sep 07 11:58:14.115321 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.115277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97c5596d-51f5-48f2-ac89-f09df1ef17c9-metrics-certs\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.115525 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.115364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97c5596d-51f5-48f2-ac89-f09df1ef17c9-openshift-service-ca\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.115525 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.115389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jjv7\" (UniqueName: \"kubernetes.io/projected/97c5596d-51f5-48f2-ac89-f09df1ef17c9-kube-api-access-5jjv7\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.115525 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.115434    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97c5596d-51f5-48f2-ac89-f09df1ef17c9-maas-api-tls\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.216868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.216833    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97c5596d-51f5-48f2-ac89-f09df1ef17c9-openshift-service-ca\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.216868 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.216868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jjv7\" (UniqueName: \"kubernetes.io/projected/97c5596d-51f5-48f2-ac89-f09df1ef17c9-kube-api-access-5jjv7\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.217138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.216904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97c5596d-51f5-48f2-ac89-f09df1ef17c9-maas-api-tls\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.217138 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.216970    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97c5596d-51f5-48f2-ac89-f09df1ef17c9-metrics-certs\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.217506 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.217477    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97c5596d-51f5-48f2-ac89-f09df1ef17c9-openshift-service-ca\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.219555 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.219526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97c5596d-51f5-48f2-ac89-f09df1ef17c9-metrics-certs\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.219650 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.219633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97c5596d-51f5-48f2-ac89-f09df1ef17c9-maas-api-tls\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.223925 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.223900    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jjv7\" (UniqueName: \"kubernetes.io/projected/97c5596d-51f5-48f2-ac89-f09df1ef17c9-kube-api-access-5jjv7\") pod \"maas-api-75d79b99ff-nzm67\" (UID: \"97c5596d-51f5-48f2-ac89-f09df1ef17c9\") " pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.336042 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.335933    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:14.612595 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:14.612563    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"]
Sep 07 11:58:14.615430 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:58:14.615400    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c5596d_51f5_48f2_ac89_f09df1ef17c9.slice/crio-2debf21e90c7cd4baa197df60971328e889a54f75625f78684912ccd3b75f4bf WatchSource:0}: Error finding container 2debf21e90c7cd4baa197df60971328e889a54f75625f78684912ccd3b75f4bf: Status 404 returned error can't find the container with id 2debf21e90c7cd4baa197df60971328e889a54f75625f78684912ccd3b75f4bf
Sep 07 11:58:15.469378 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:15.469342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67" event={"ID":"97c5596d-51f5-48f2-ac89-f09df1ef17c9","Type":"ContainerStarted","Data":"2debf21e90c7cd4baa197df60971328e889a54f75625f78684912ccd3b75f4bf"}
Sep 07 11:58:16.476302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:16.476204    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67" event={"ID":"97c5596d-51f5-48f2-ac89-f09df1ef17c9","Type":"ContainerStarted","Data":"5ca1d2ed02252d688dce385e5dffbfd09c9424d937028a7abb3fcaad4120fab3"}
Sep 07 11:58:16.476302 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:16.476286    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:16.493519 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:16.493463    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67" podStartSLOduration=0.886700687 podStartE2EDuration="2.49344483s" totalImagesPullingTime="1.606744143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:58:16.492800568 +0000 UTC m=+2128.440151351" watchObservedRunningTime="2026-09-07 11:58:16.49344483 +0000 UTC m=+2128.440795616"
Sep 07 11:58:22.486646 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:22.486610    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75d79b99ff-nzm67"
Sep 07 11:58:45.212451 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:45.212362    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f74dc8dd5-h6hdc_61fad53d-2aa4-42a6-99bf-da92e7a425f0/manager/0.log"
Sep 07 11:58:45.325602 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:45.325571    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-89plz_3118cf26-0bbc-4feb-96da-ba41f9a00b00/server/0.log"
Sep 07 11:58:45.447296 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:45.447264    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h9tlh_2550995d-e8a3-4c74-9b2e-345f9ce98640/manager/0.log"
Sep 07 11:58:45.829597 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:45.829568    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f479d7474-pvz66_00c310e0-8385-4e86-9bb2-96e92a5ab804/copy-manifests/0.log"
Sep 07 11:58:45.841515 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:45.841482    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f479d7474-pvz66_00c310e0-8385-4e86-9bb2-96e92a5ab804/manager/0.log"
Sep 07 11:58:46.198443 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:46.198361    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-75d79b99ff-nzm67_97c5596d-51f5-48f2-ac89-f09df1ef17c9/maas-api/0.log"
Sep 07 11:58:46.559384 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:46.559352    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sjzrr_5b48c858-4ea8-4268-9c58-568ba90f312c/postgres/0.log"
Sep 07 11:58:47.656346 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:47.656312    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-4xmxz_edf3f957-8a75-445e-8780-be269da59b07/manager/0.log"
Sep 07 11:58:48.172223 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:48.172195    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-d5tkg_7854a100-6825-45b0-b860-c4db62d2fe47/manager/0.log"
Sep 07 11:58:48.565073 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:48.565040    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch_c7642862-5cb6-44a0-9100-311f0a5f9156/istio-proxy/0.log"
Sep 07 11:58:48.967179 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:48.967084    2560 scope.go:117] "RemoveContainer" containerID="31366058b2c401d9af6e8b82a9faf7cbeea7e3113e9d4658c0d44e0dfccbe20e"
Sep 07 11:58:49.051502 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:49.051472    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/1.log"
Sep 07 11:58:49.171891 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:49.171865    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hjfht_8d9a6776-8a2f-4f40-94b3-3b3c8f79de22/payload-pre-processing/0.log"
Sep 07 11:58:49.310202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:49.310163    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pm4dl_bbc264ca-820b-46cc-974f-e6feceb75d72/payload-processing/0.log"
Sep 07 11:58:50.046063 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:50.046035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-vrsg6_a2d451f4-150b-4384-bdbb-897538778d55/main/0.log"
Sep 07 11:58:50.163470 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:50.163439    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-6rgp4_ed4b4513-6197-4fa6-a2f7-efe00fbc75a9/main/0.log"
Sep 07 11:58:50.397866 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:50.397786    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ljndv_1a091f58-26cb-42cb-8920-671613c6b0e3/main/0.log"
Sep 07 11:58:59.854281 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:58:59.854244    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6dsdg_62b756ad-9c5d-493f-9afa-553c4a741503/global-pull-secret-syncer/0.log"
Sep 07 11:59:00.010517 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:00.010483    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lwmzh_8d3cbf66-5aad-433f-9f7a-79dc9c747517/konnectivity-agent/0.log"
Sep 07 11:59:00.077108 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:00.077064    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-45.ec2.internal_422a2dc891c89e8aeb801cae87a1e08b/haproxy/0.log"
Sep 07 11:59:04.591354 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:04.591325    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-4xmxz_edf3f957-8a75-445e-8780-be269da59b07/manager/0.log"
Sep 07 11:59:04.740013 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:04.739986    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-d5tkg_7854a100-6825-45b0-b860-c4db62d2fe47/manager/0.log"
Sep 07 11:59:06.485069 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.485044    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87lq8_60e7116d-8c77-473c-95ed-04e86c9e7705/node-exporter/0.log"
Sep 07 11:59:06.508637 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.508616    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87lq8_60e7116d-8c77-473c-95ed-04e86c9e7705/kube-rbac-proxy/0.log"
Sep 07 11:59:06.526874 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.526848    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87lq8_60e7116d-8c77-473c-95ed-04e86c9e7705/init-textfile/0.log"
Sep 07 11:59:06.776766 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.776665    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/prometheus/0.log"
Sep 07 11:59:06.796198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.796173    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/config-reloader/0.log"
Sep 07 11:59:06.813475 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.813454    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/thanos-sidecar/0.log"
Sep 07 11:59:06.832168 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.832148    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/kube-rbac-proxy-web/0.log"
Sep 07 11:59:06.854505 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.854478    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/kube-rbac-proxy/0.log"
Sep 07 11:59:06.908216 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.908190    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/kube-rbac-proxy-thanos/0.log"
Sep 07 11:59:06.955693 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:06.955669    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_75ca6d96-a319-483a-bfd2-aade442e5022/init-config-reloader/0.log"
Sep 07 11:59:08.792874 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:08.792845    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-8lnf8_151e67a0-7edf-4a1a-a0a0-d16fc7ded050/networking-console-plugin/0.log"
Sep 07 11:59:09.853345 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.853226    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"]
Sep 07 11:59:09.856532 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.856507    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:09.858803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.858780    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9f2qg\"/\"kube-root-ca.crt\""
Sep 07 11:59:09.858803 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.858792    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9f2qg\"/\"openshift-service-ca.crt\""
Sep 07 11:59:09.858986 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.858786    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9f2qg\"/\"default-dockercfg-svqxd\""
Sep 07 11:59:09.868222 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.868188    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"]
Sep 07 11:59:09.948767 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.948709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-sys\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:09.948957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.948785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-lib-modules\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:09.948957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.948843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-podres\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:09.948957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.948897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqgh\" (UniqueName: \"kubernetes.io/projected/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-kube-api-access-dwqgh\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:09.948957 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.948957    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-proc\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:09.966834 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:09.966808    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77886d4955-865xr_5bf0a0e9-d068-44c9-a859-a86ae59865b9/console/0.log"
Sep 07 11:59:10.050006 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.049969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqgh\" (UniqueName: \"kubernetes.io/projected/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-kube-api-access-dwqgh\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050032    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-proc\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050061    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-sys\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-lib-modules\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050163    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-podres\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-proc\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050202 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050186    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-sys\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050440 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-lib-modules\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.050440 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.050294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-podres\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.059518 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.059059    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqgh\" (UniqueName: \"kubernetes.io/projected/66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61-kube-api-access-dwqgh\") pod \"perf-node-gather-daemonset-57q6j\" (UID: \"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61\") " pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.167289 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.167182    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.440113 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.440063    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"]
Sep 07 11:59:10.442476 ip-10-0-138-45 kubenswrapper[2560]: W0907 11:59:10.442444    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod66d0c1a2_e4d4_4eac_8caa_25fbcbfbfd61.slice/crio-93cfb45d879851591202895ead86f4de4d610fac298589d5396e0e23cfe758a8 WatchSource:0}: Error finding container 93cfb45d879851591202895ead86f4de4d610fac298589d5396e0e23cfe758a8: Status 404 returned error can't find the container with id 93cfb45d879851591202895ead86f4de4d610fac298589d5396e0e23cfe758a8
Sep 07 11:59:10.734308 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.734270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j" event={"ID":"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61","Type":"ContainerStarted","Data":"5650f4a651e6924e24ae4ebd3b7c8806eb7a64a7ea71567284d60728653888b9"}
Sep 07 11:59:10.734496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.734326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j" event={"ID":"66d0c1a2-e4d4-4eac-8caa-25fbcbfbfd61","Type":"ContainerStarted","Data":"93cfb45d879851591202895ead86f4de4d610fac298589d5396e0e23cfe758a8"}
Sep 07 11:59:10.734496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.734402    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:10.745502 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.745478    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-54rgr_60e08d47-09d8-49fd-a7a5-afe61305526d/volume-data-source-validator/0.log"
Sep 07 11:59:10.751591 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:10.751289    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j" podStartSLOduration=1.751272548 podStartE2EDuration="1.751272548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:59:10.750915144 +0000 UTC m=+2182.698265942" watchObservedRunningTime="2026-09-07 11:59:10.751272548 +0000 UTC m=+2182.698623344"
Sep 07 11:59:11.909403 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:11.909374    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q7r6q_84e6938a-fe71-410a-aa30-9a929b1be9a9/dns/0.log"
Sep 07 11:59:11.926824 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:11.926792    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q7r6q_84e6938a-fe71-410a-aa30-9a929b1be9a9/kube-rbac-proxy/0.log"
Sep 07 11:59:11.987259 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:11.987230    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fjx89_d0c0a647-41c3-4e09-8060-ad1ae7940794/dns-node-resolver/0.log"
Sep 07 11:59:12.532829 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:12.532800    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7968944dff-4prpl_f1131812-e49c-4586-98d5-efbedc41302d/registry/0.log"
Sep 07 11:59:12.575083 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:12.575060    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q4gvm_4f8d8134-6363-4b59-81c9-9b01c1b2df54/node-ca/0.log"
Sep 07 11:59:13.566561 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:13.566531    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch_c7642862-5cb6-44a0-9100-311f0a5f9156/istio-proxy/0.log"
Sep 07 11:59:13.919470 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:13.919399    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/0.log"
Sep 07 11:59:13.929182 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:13.929160    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/1.log"
Sep 07 11:59:13.964436 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:13.964397    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hjfht_8d9a6776-8a2f-4f40-94b3-3b3c8f79de22/payload-pre-processing/0.log"
Sep 07 11:59:13.997293 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:13.997257    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pm4dl_bbc264ca-820b-46cc-974f-e6feceb75d72/payload-processing/0.log"
Sep 07 11:59:14.877928 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:14.877896    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sdzvg_30549a7f-5111-45f6-ba9f-13729a5d9322/serve-healthcheck-canary/0.log"
Sep 07 11:59:15.423105 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:15.423065    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4lks4_bcd1d32e-7641-4e8b-844b-9e8f1531ea8c/kube-rbac-proxy/0.log"
Sep 07 11:59:15.441799 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:15.441765    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4lks4_bcd1d32e-7641-4e8b-844b-9e8f1531ea8c/exporter/0.log"
Sep 07 11:59:15.459386 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:15.459357    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4lks4_bcd1d32e-7641-4e8b-844b-9e8f1531ea8c/extractor/0.log"
Sep 07 11:59:16.749149 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:16.749121    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9f2qg/perf-node-gather-daemonset-57q6j"
Sep 07 11:59:17.773137 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:17.773100    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f74dc8dd5-h6hdc_61fad53d-2aa4-42a6-99bf-da92e7a425f0/manager/0.log"
Sep 07 11:59:17.796940 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:17.796912    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-89plz_3118cf26-0bbc-4feb-96da-ba41f9a00b00/server/0.log"
Sep 07 11:59:17.869417 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:17.869384    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h9tlh_2550995d-e8a3-4c74-9b2e-345f9ce98640/manager/0.log"
Sep 07 11:59:18.070843 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:18.070729    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f479d7474-pvz66_00c310e0-8385-4e86-9bb2-96e92a5ab804/manager/0.log"
Sep 07 11:59:18.086078 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:18.086049    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f479d7474-pvz66_00c310e0-8385-4e86-9bb2-96e92a5ab804/copy-manifests/0.log"
Sep 07 11:59:19.512097 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:19.512070    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-79b5d8b54d-s6pz6_1046905b-f697-422f-96ad-75be2d3a9ff8/manager/0.log"
Sep 07 11:59:20.104471 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:20.104386    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch_c7642862-5cb6-44a0-9100-311f0a5f9156/istio-proxy/0.log"
Sep 07 11:59:20.433774 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:20.433678    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/0.log"
Sep 07 11:59:20.443751 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:20.443715    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/1.log"
Sep 07 11:59:20.467585 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:20.467554    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hjfht_8d9a6776-8a2f-4f40-94b3-3b3c8f79de22/payload-pre-processing/0.log"
Sep 07 11:59:20.494574 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:20.494547    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pm4dl_bbc264ca-820b-46cc-974f-e6feceb75d72/payload-processing/0.log"
Sep 07 11:59:22.010161 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:22.010125    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xh8ch_c7642862-5cb6-44a0-9100-311f0a5f9156/istio-proxy/0.log"
Sep 07 11:59:22.370792 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:22.370696    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/0.log"
Sep 07 11:59:22.381099 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:22.381074    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qnwdb_92a89314-0821-4934-a99c-74b3a9f3e3eb/istio-proxy/1.log"
Sep 07 11:59:22.409058 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:22.409026    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hjfht_8d9a6776-8a2f-4f40-94b3-3b3c8f79de22/payload-pre-processing/0.log"
Sep 07 11:59:22.433570 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:22.433539    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pm4dl_bbc264ca-820b-46cc-974f-e6feceb75d72/payload-processing/0.log"
Sep 07 11:59:24.573689 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:24.573659    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-knz5r_b456dadd-b2de-4b84-99d8-69785d6ecafa/migrator/0.log"
Sep 07 11:59:24.593837 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:24.593806    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-knz5r_b456dadd-b2de-4b84-99d8-69785d6ecafa/graceful-termination/0.log"
Sep 07 11:59:26.158578 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.158550    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/kube-multus-additional-cni-plugins/0.log"
Sep 07 11:59:26.176198 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.176173    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/egress-router-binary-copy/0.log"
Sep 07 11:59:26.192947 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.192919    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/cni-plugins/0.log"
Sep 07 11:59:26.209516 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.209490    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/bond-cni-plugin/0.log"
Sep 07 11:59:26.226132 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.226104    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/routeoverride-cni/0.log"
Sep 07 11:59:26.243802 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.243771    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/whereabouts-cni-bincopy/0.log"
Sep 07 11:59:26.263597 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.263569    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrp82_e2760e4f-b854-488a-9886-e92ade779208/whereabouts-cni/0.log"
Sep 07 11:59:26.373679 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.373651    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lhhhk_4f5ad469-1623-4329-a48d-e71071315cdd/kube-multus/0.log"
Sep 07 11:59:26.424496 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.424417    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bjfn8_9e39c68a-5691-47a5-bb3c-fa289e615054/network-metrics-daemon/0.log"
Sep 07 11:59:26.440561 ip-10-0-138-45 kubenswrapper[2560]: I0907 11:59:26.440535    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bjfn8_9e39c68a-5691-47a5-bb3c-fa289e615054/kube-rbac-proxy/0.log"